Continuous Optimal Control Research Articles

New Equilibria and Dynamic Structures Under Continuous Optimal Feedback Control

In this paper, the new equilibria realized by continuous optimal control inputs and the dynamic structure around them are studied. Using the Euler–Lagrange equation, which is a necessary condition for optimal control problems, the equations of motion of a dynamic system with optimal control inputs that minimize the quadratic cost function are described in terms of state and adjoint variables. Based on the equations of motion, equilibrium conditions are derived, and the properties of equilibria are analyzed for the two-body and Hill three-body problems. The stability and dynamic structure around unstable equilibria are also characterized to get insights into the properties of optimal trajectories.

Reentry trajectory planning for hypersonic vehicles via an improved sequential convex programming method

The reentry trajectory planning problem of hypersonic vehicles is generally a continuous and nonconvex optimization problem. The solution efficiency and convergence properties of the planning algorithm become an essential point when considering the practical application scenarios. In this paper, an augmented-Lagrange-based improved sequential convex programming algorithm is proposed to solve it and achieve superior solutions with higher accuracy and efficiency. Firstly, the mapped Chebyshev pseudo-spectral method is utilized to convert the continuous optimal control problem into a highly approximately discretized one. In this framework, the discretization technique is employed to discretize the states and control variables at the time interval, which leads to the improvement of ill-conditioned differential matrix and better approximation properties of the variables. Moreover, by introducing slack variables, automatically updated penalty parameters and Lagrange multipliers during the convexification processing, the original nonconvex problem is transformed into an augmented-Lagrange-function-formed relaxed convex one. Then, the arrived convex subproblem can be solved iteratively by the matured interior-point method until the predetermined precision is satisfied. Based on this transformation, it can effectively alleviate the numerical difficulty and improve the convergence property of the proposed improved sequential convex programming algorithm. Finally, Rigorous theoretical analysis is conducted to prove that the solution sequence generated by the proposed algorithm could converge to an approximate global solution of the original problem. Numerical simulations are also presented, and it is demonstrated that the solution time of the proposed algorithm is more than 95% less than that of the traditional nonlinear programming methods without losing the merits of high precision.

On the adaptation of the Lagrange formalism to continuous time stochastic optimal control: A Lagrange-Chow redux

We show how the classical Lagrangian approach to solving constrained optimization problems from standard calculus can be extended to solve continuous time stochastic optimal control problems. Connections to mainstream approaches such as the Hamilton-Jacobi-Bellman equation and the stochastic maximum principle are drawn. Our approach is linked to the stochastic maximum principle, but more direct and tied to the classical Lagrangian principle, avoiding the use of backward stochastic differential equations in its formulation. Using infinite dimensional functional analysis, we formalize and extend the approach first outlined in Chow (1992) within a rigorous mathematical setting using infinite dimensional functional analysis. We provide examples that demonstrate the usefulness and effectiveness of our approach in practice. Further, we demonstrate the potential for numerical applications facilitating some of our key equations in combination with Monte Carlo backward simulation and linear regression, therefore illustrating a completely different and new avenue for the numerical application of Chow's methods.

Quaternary Continuous Classical Boundary Optimal Control Problem Dominating by Parabolic System

In this paper, our purpose is to study the quaternary continuous classical boundary optimal control vector problem dominated by a quaternary linear parabolic boundary value problem. Under suitable assumptions and with a given quaternary continuous classical boundary control vector, the existence theorem for a unique quaternary state vector solution to the weak form is stated and demonstrated via the method of Galerkin. Furthermore, the continuity of the Lipschitz operator between the state vector solution to the weak form for the dominating equations and the corresponding are proved. The existence of a quaternary continuous classical boundary optimal control vector is stated and demonstrated under suitable Assumptions. The mathematical formulation for the quaternary adjoint boundary value problem associated with each considered boundary value problem is obtained and the Frѐchet derivative for the objective function is derived. Finally, the necessary conditions for the optimality theorem of the problem are stated and demonstrated.

Energy-Based Continuous Inverse Optimal Control.

The problem of continuous inverse optimal control (over finite time horizon) is to learn the unknown cost function over the sequence of continuous control variables from expert demonstrations. In this article, we study this fundamental problem in the framework of energy-based model (EBM), where the observed expert trajectories are assumed to be random samples from a probability density function defined as the exponential of the negative cost function up to a normalizing constant. The parameters of the cost function are learned by maximum likelihood via an "analysis by synthesis" scheme, which iterates: 1) synthesis step: sample the synthesized trajectories from the current probability density using the Langevin dynamics via backpropagation through time and 2) analysis step: update the model parameters based on the statistical difference between the synthesized trajectories and the observed trajectories. Given the fact that an efficient optimization algorithm is usually available for an optimal control problem, we also consider a convenient approximation of the above learning method, where we replace the sampling in the synthesis step by optimization. Moreover, to make the sampling or optimization more efficient, we propose to train the EBM simultaneously with a top-down trajectory generator via cooperative learning, where the trajectory generator is used to fast initialize the synthesis step of the EBM. We demonstrate the proposed methods on autonomous driving tasks and show that they can learn suitable cost functions for optimal control.

Solvability of the Quaternary Continuous Classical Boundary Optimal Control Dominated by Quaternary Parabolic System

The purpose of this paper is to study the solvability of the quaternary continuous classical boundary optimal control vector problem dominated by quaternary nonlinear parabolic boundary value problem with state constraints. The existence theorem for a quaternary continuous classical boundary optimal control vector with equality and inequality state constraints is stated and demonstrated under suitable assumptions. The mathematical formulation of the quaternary adjoint eqs. associated with the quaternary nonlinear parabolic boundary value problem with state constraints is discovered. The Fréchet derivative of the cost function and the constraints functions are derived. The necessary and sufficient theorems (conditions) for optimality of the quaternary continuous classical boundary optimal control vector problem are stated and demonstrated under suitable assumptions.

Quaternary Boundary Optimal Control Problem Dominating by Quaternary Nonlinear Parabolic System

In this paper, our purpose is to study the quaternary continuous classical boundary optimal control vector problem (QCCBOCVP) dominated by the quaternary nonlinear parabolic boundary value problem (QNLPBVP). Under suitable assumptions and with given quaternary continuous classical boundary control vector (QCCBCV), the existence theorem for a unique quaternary state vector solution (QSVS) of the weak form (WF) for the QNLPBVP is stated and demonstrated via the Method of Galerkin and the first compactness theorem. Furthermore, the continuity of the Lipchitz operator between the QSVS of the WF for the QLPBVP and the corresponding QCCBCV is proved. The existence of a quaternary continuous classical boundary optimal control vector (QCCBOVC) is stated and demonstrated under suitable assumptions.

Optimal continuous control of refrigerator for electricity cost minimization—Hierarchical reinforcement learning approach

A refrigerator is a commonly used household appliance; however, limited research has focused on optimizing temperature control policy with the consideration of Time-of-Use (ToU) electricity price. This paper introduces a novel framework that utilizes hierarchical reinforcement learning (HRL) to control the intensity of refrigerator motors. The objective is to achieve both temperature regulation and cost savings under ToU and stochastic usage patterns. The problem is tackled by two HRL agents. The high-level agent is responsible for determining temperature reference based on ToU pricing, while the low-level agent adjusts the motor intensity to meet this temperature reference. To tackle non-stationarity in HRL, the high-level agent employs hindsight action transition and reward function approximation, while the low-level agent employs hindsight goal transition. Through the experimental evaluation, we found that the proposed method outperforms the conventional control methods and standard reinforcement learning approaches. It achieves the lowest total costs, resulting in a significant cost reduction of 5%–24%.

Approximate Solution for Some Continuous Optimal Control Problems Using Veita-Pell Polynomials

In this paper, Veita-Pell polynomials (VPPs) are first presented with some interesting properties. These properties are utilized to construct a general explicit formula of their operational matrix of derivative. Then an appropriate direct technique is suggested for solving quadratic optimal control problems based on VPPs and the idea of state parameterization algorithm. The resulting performance index optimal value shows the proposed method is able to provide a good treatment with fast convergence. The effectiveness of the presented method is illustrated by solving three numerical examples. The obtained results show that as the number of basis functions VPPs increase the error in the solution by the present method will be decreased and it may exactly close with the analytical one. This is the main modification of the algorithm and this contribution in the using special basis functions in obtaining an approximate solution with minimum number of VPPs and satisfactory accuracy.

Classical Continuous Constraint Boundary Optimal Control Vector Problem for Triple Nonlinear Parabolic System

In this paper, our purpose is to study the classical continuous constraints boundary optimal triple control vector problem dominating nonlinear triple parabolic boundary value problem. The existence theorem for a classical continuous triple optimal control vector CCCBOTCV is stated and proved under suitable assumptions. The mathematical formulation of the adjoint triple boundary value problem associated with the nonlinear triple parabolic boundary value problem is discovered. The Fréchet derivative of the Hamiltonian is derived. Under proper assumptions, both theorems are granted; the necessary conditions for optimality and the sufficient conditions for optimality of the classical continuous constraints boundary optimal triple control vector problem are stated and proven.

Classical Continuous Boundary Optimal Control Vector Problem for Triple Nonlinear Parabolic System

In this paper, our purpose is to study the classical continuous boundary optimal triple control vector problem (CCBOTCVP) dominating by nonlinear triple parabolic boundary value problem (NLTPBVP). Under suitable assumptions and with given classical continuous boundary triple control vector (CCBTCV), the existence theorem for a unique state triple vector solution (STVS) of the weak form W.F for the NLTPBVP is stated and demonstrated via the Method of Galerkin (MGa), and the first compactness theorem. Furthermore, the continuity operator between the STVS of the WFO for the NLTPBVP and the corresponding CCBTCV is stated and demonstrated. The continuity of the Lipschitz (LIP.) operator between the STVS of the WFo for the QNLPBVP and the corresponding CCBTCV is proved. The existence of a CCBOTCV is stated and demonstrated under suitable conditions.

Trajectory planning for satellite cluster reconfigurations with sequential convex programming method

This paper presents trajectory planning algorithms for large-scale satellite clusters reconfiguration based on sequential convex programming, and these algorithms consider fuel consumption and collision avoidance. Firstly, the trajectory planning problem is formulated as a nonconvex optimal control problem with nonlinear dynamics and nonconvex path constraints. Secondly, the original nonlinear continuous optimal control problem is transformed into a discrete convex optimization subproblem through linearization and discretization. Collision avoidance strategy between discrete points and obstacle avoidance constraints are considered in the convex subproblem to ensure that the trajectories are collision-free. Thirdly, coupled and decoupled sequential convex programming methods are proposed for rapidly generating collision-free and fuel-efficient trajectories. Finally, comparative numerical simulations are presented to demonstrate that as the number of satellites increases, 94 to 99 percent performance improvement over the pseudo-spectral method is achieved in terms of computational efficiency.

Adaptive control of resource flow to optimize construction work and cash flow via online deep reinforcement learning

Due to complexity and dynamics of construction work, resource, and cash flows, poor management of them usually leads to time and cost overruns, bankruptcy, even project failure. Existing approaches in construction failed to achieve optimal control of resource flow in a dynamic environment with uncertainty. Therefore, this paper proposes a model and method to adaptive control the resource flows to optimize the work and cash flows of construction projects. First, a mathematical model based on a partially observable Markov decision process is established to formulate the complex interactions of construction work, resource, and cash flows as well as uncertainty and variability of diverse influence factors. Meanwhile, to efficiently find the optimal solutions, a Deep Reinforcement Learning (DRL) based method is introduced to realize the continuous adaptive optimal control of labor and material flows, thereby optimizing the work and cash flows. To assist the training process of DRL, a simulator based on discrete event simulation is also developed to mimic the dynamic features and external environments of a project. Experiments in simulated scenarios illustrate that our method outperforms the vanilla empirical method and genetic algorithm, possesses remarkable capability in diverse projects and external environments, and a hybrid agent of DRL and empirical method leads to the best result. This paper contributes to adaptive control and optimization of coupled work, resource, and cash flows, and may serve as a step stone for adopting DRL technology in construction project management.

The Classical Continuous Optimal Control for Quaternary parabolic boundary value problem

The aim of this paper is to study the quaternary classical continuous optimal control for a quaternary linear parabolic boundary value problems(QLPBVPs). The existence and uniqueness theorem of the continuous quaternary state vector solution for the weak form of the QLPBVPs with given quaternary classical continuous control vector (QCCCV) is stated and proved via the Galerkin Method. In addition, the existence theorem of a quaternary classical continuous optimal control vector governinig by the QLPBVPs is stated and demonstrated. The Fréchet derivative for the cost function is derived. Finally, the necessary conditions for the optimality theorem of the proposed problem is stated and demonstrated.

Numerical analysis of a family of simultaneous distributed-boundary mixed elliptic optimal control problems and their asymptotic behaviour through a commutative diagram and error estimates

In this paper, we consider a family of simultaneous distributed-boundary optimal control problems (Pα) on the internal energy and the heat flux for a system governed by a mixed elliptic variational equality with a parameter α>0 (the heat transfer coefficient on a portion of the boundary of the domain) and a simultaneous distributed-boundary optimal control problem (P) governed also by an elliptic variational equality with a Dirichlet boundary condition on the same portion of the boundary. We formulate discrete approximations Phα and Ph of the optimal control problems Pα and (P) respectively, for each h>0 and for each α>0, through the finite element method with Lagrange’s triangles of type 1 with parameter h (the longest side of the triangles). The goal of this paper is to study the convergence of this family of discrete simultaneous distributed-boundary mixed elliptic optimal control problems Phα when the parameters α goes to infinity and the parameter h goes to zero simultaneously. We prove the convergence of the family of discrete problems Phα to the discrete problem Ph when α→+∞, for each h>0, in adequate functional spaces. We study the convergence of the discrete problems Phα and Ph, for each α>0, when h→0+ obtaining a commutative diagram which relates the continuous and discrete simultaneous distributed-boundary mixed elliptic optimal control problems Phα,Pα,Ph and (P) by taking the limits h→0+ and α→+∞ respectively. We also study the double convergence of Phα to (P) when (h,α)→(0+,+∞) which represents the diagonal convergence in the above commutative diagram.

The Classical Continuous Optimal Control for Quaternary Nonlinear Parabolic Boundary Value Problems

In this paper, our purpose is to study the classical continuous optimal control (CCOC) for quaternary nonlinear parabolic boundary value problems (QNLPBVPs). The existence and uniqueness theorem (EUTh) for the quaternary state vector solution (QSVS) of the weak form (WF) for the QNLPBVPs with a given quaternary classical continuous control vector (QCCCV) is stated and proved via the Galerkin Method (GM) and the first compactness theorem under suitable assumptions(ASSUMS). Furthermore, the continuity operator for the existence theorem of a QCCCV dominated by the QNLPBVPs is stated and proved under suitable conditions.

Hurwicz criterion based optimal control model for uncertain descriptor systems with an application to industrial management

In this work, optimal control models subject to discrete and continuous uncertain descriptor systems are both investigated under Hurwicz criterion. At first, by employing dynamic programming method, a recurrence equation is obtained to simplify the discrete optimal control model, and a bang-bang control problem is settled according to this equation. Then, for the continuous optimal control model, an equation of optimality is proposed and testified with the help of uncertainty theory and the principle of optimality, revealing the internal connection between optimal control and optimal value. Finally, a numerical example and an industrial management model are provided to illustrate the validness of above results.

Optimal Safety for Constrained Differential Inclusions Using Nonsmooth Control Barrier Functions

For a broad class of nonlinear systems, we formulate the problem of guaranteeing safety with optimality under constraints. Specifically, we define controlled safety for differential inclusions with constraints on the states and the inputs. Through the use of nonsmooth analysis tools, we show that a continuous optimal control law can be selected from a set-valued constraint capturing the system constraints and conditions guaranteeing safety using control barrier functions. Our results guarantee optimality and safety via a continuous state-feedback law designed using nonsmooth control barrier functions. An example pertaining to obstacle avoidance with a target illustrates our results and the associated benefits of using nonsmooth control barrier functions.

Physical-model-free intelligent energy management for a grid-connected hybrid wind-microturbine-PV-EV energy system via deep reinforcement learning approach

Renewable -based microgrid (MG) is recognized as an eco-friendly solution in the development of renewable energy (RE). Moreover, the MG energy management with high RE penetration faces complicate uncertainties due to the inaccuracy of predictions. Besides, the growing participation of electric vehicles (EVs) makes the traditional model-based methods even more infeasible. Considering uncertainties associated with RE, EVs, and electricity price, a model-free deep reinforcement learning (DRL), namely twin delayed deep deterministic policy gradient (TD3), is employed to develop an optimized control strategy to minimize operating costs and satisfy charging expectations. The energy management problem is first formulated as a Markov decision process. Then, TD3 solely relies on the limited observation to find the optimal continuous control strategy. The proposed method flexibly adjusts the operating and charging strategies of components according to RE output and electricity price. Its real-time optimized performance on three consecutive days along with electricity price is evaluated, indicating its practical potential for future application. Additionally, comparison results demonstrate that the proposed method reduces the total costs up to 15.27% and 4.24%, respectively, compared to traditional optimization and other DRL methods, which illustrates the superiority of the TD3 method on optimizing total costs of the considered MG.

Bi-objective design-for-control for improving the pressure management and resilience of water distribution networks

This paper investigates control and design-for-control strategies to improve the resilience of sectorized water distribution networks (WDN), while minimizing pressure induced pipe stress and leakage. Both evolutionary algorithms (EA) and gradient-based mathematical optimization approaches are investigated for the solution of the resulting large-scale non-linear (NLP) and bi-objective mixed-integer non-linear programs (BOMINLP). While EAs have been successfully applied to solve discrete network design problems for large-scale WDNs, gradient-based mathematical optimization methods are more computationally efficient when dealing with large search spaces associated with continuous variables in optimal network control problems. Considering the advantages of each method, we propose a sequential hybrid method for the optimal design-for-control of large-scale WDNs, where a refinement stage relying on gradient-based mathematical optimization is used to solve continuous optimal control problems corresponding to design solutions returned by an initial EA search. The proposed method is applied to compute the Pareto front of a bi-objective design-for-control problem for the operational network BWPnet, where we consider reopening closed connections between isolated supply areas. The results show that the considered design-for-control strategy increases the resilience of BWPnet while minimizing pressure induced leakage. Moreover, the refinement stage of the proposed hybrid method efficiently improves the coarse approximation computed by the initial EA search, returning a continuous and even Pareto front approximation.