Equivalent Optimal Control Problem Research Articles

The method of determining optimal control of the thermoelastic state of piece-homogeneous body using a stationary temperature field

This paper proposes a new highly effective method for determining the optimal control of the stress-strain state of spatially multi-connected composite bodies using a stationary temperature field. The proposed method is considered based on the example of a stationary axisymmetric thermoelastic problem for a space with a spherical inclusion and cavity. The proposed method is based on the generalized Fourier method and reduces the original problem to an equivalent problem of optimal control, in which the state of the object is determined by an infinite system of linear algebraic equations, the right-hand side of which parametrically depends on the control. At the same time, the functional of the cost of the initial problem is transformed into a quadratic functional, which depends on the state of the equivalent system and parametrically on the control. The limitation on the temperature distribution is replaced by the value of the control norm in the space of square summable sequences. In fact, this paper considers for the first time the problem of optimal control of an infinite system of linear algebraic equations and develops a method for its solution. The proposed method is based on presenting the solutions of infinite systems in a parametric form, which makes it possible to reduce equivalent problem to the problem of conditional extremum of a quadratic functional, which explicitly depends on the control. A further solution to this problem A further solution to this problem is found by the Lagrange method using the spectral decomposition of the quadratic functional matrix. found by the Lagrange method using the spectral decomposition of the quadratic functional matrix. The method developed in this paper is strictly justified. For all infinite systems, the Fredholm property of their operators is proved. As an important result necessary for substantiation, for the first time, an estimate from below of the module of the multi-parameter determinant of the resolving system of the boundary value problem of conjugation – space with a spherical inclusion – was obtained when solving it using the Fourier method. The theorem that establishes the conditions for the existence and uniqueness of the solution of equivalent problem or optimal control problem without restrictions in the space of square summable sequences is proved. The numerical algorithm is based on a reduction method for infinite systems of linear algebraic equations. Estimates of the practical accuracy of the numerical algorithm demonstrated the stability of the method and sufficiently high accuracy even with close location of the boundary surfaces. Graphs showing the optimal temperature distribution for various geometric parameters of the problem and their analysis are provided. The proposed method extends to boundary value problems with different geometries.

Adaptive fault-tolerant optimal control for hypersonic vehicles with state constrains based on adaptive dynamic programming

This paper addresses a novel adaptive fault-tolerant control (FTC) design based on adaptive dynamic programming (ADP) technique for hypersonic vehicle (HSV) subject to actuator fault and state constrains. The total control input is constructed by the combination of backstepping-based adaptive FTC and the ADP-based adaptive optimal control to improve the tracking performance and fault-tolerant capacity. By introducing a barrier Lyapunov function (BLF) to deal with the state constraints, a backstepping-based FTC scheme is designed to transform the tracking control problem into an equivalent optimal control problem. Subsequently, an adaptive optimal control strategy is developed by using the ADP technique to provide a supplementary control action. A critic network is constructed to solve the Hamilton–Jacobi–Bellman (HJB) equation online. The convergence properties of the closed-loop system are developed by utilizing the Lyapunov stability theory. Finally, simulation results are carried out to illustrate the effectiveness and efficiency of the proposed adaptive ADP-based FTC strategy.

Trajectory tracking control for underactuated autonomous vehicles via adaptive dynamic programming

In this paper, the trajectory tracking control problem for underactuated autonomous vehicles moving in 2-D space is studied. Firstly, the trajectory tracking error dynamics with the equivalent point being the origin are given. Since both model and input uncertainties of vehicles are considered, a robust control problem is formulated by giving the extended trajectory tracking error dynamics. Through showing that a robust control law is equivalent to an optimal control law for a nominal system under some conditions, the robust control problem is transformed into an equivalent optimal control problem and a cost function composed of uncertainty bounds and discount factor is constructed. In order to solve the optimal control problem, a policy iteration-based ADP algorithm is proposed, in which two neural networks, called critic and actor networks, are applied respectively to describe the unknown value function and control law. The weights of the two networks are tuned through the gradient descent method at each iteration and the optimality and convergence properties are provided in the sequel. At last, simulation results are given to illustrate the validity of the proposed algorithm.

Optimal control of the treatment and the vaccination in an epidemic switched system using polynomial approach

We study the problem of optimal switching on a treatment-vaccination model with four switches. These switches provide a more realistic approach when dealing with the control of an epidemic. We have used a relatively new approach to investigate optimal switching. The main idea behind this approach is to convert the switched system, a nonlinear, nonconvex optimal control problem, into an equivalent optimal control problem with a linear and convex structure. This allows us to obtain an equivalent convex formulation better suited to being solved by high-performance numerical computing. Toward the end, we perform some numerical simulations and draw the conclusion that the presence of treatment is essential, whereas vaccination control can be used at intervals with progressively larger gaps between the intervals.

Off-Policy Risk-Sensitive Reinforcement Learning-Based Constrained Robust Optimal Control

This article proposes an off-policy risk-sensitive reinforcement learning (RL)-based control framework to jointly optimize the task performance and constraint satisfaction in a disturbed environment. The risk-aware value function, constructed using the pseudo control and risk-sensitive input and state penalty terms, is introduced to convert the original constrained robust stabilization problem into an equivalent unconstrained optimal control problem. Then, an off-policy RL algorithm is developed to learn the approximate solution to the risk-aware value function. During the learning process, the associated approximate optimal control policy is able to satisfy both input and state constraints under disturbances. By replaying experience data to the off-policy weight update law of the critic neural network, the weight convergence is guaranteed. Moreover, online and offline algorithms are developed to serve as principled ways to record informative experience data to achieve a sufficient excitation required for the weight convergence. The proofs of system stability and weight convergence are provided. The Simulation results reveal the validity of the proposed control framework.

Uncertain optimal control problem with the first hitting time objective and application to a portfolio selection model

This paper discusses an uncertain time optimal control problem by considering time efficiency, which is to optimize the objective function about the first hitting time subject to uncertain differential equations. According to the definition of the α-path, the uncertain time optimal control problem is transformed into an equivalent deterministic optimal control problem. Two kinds of time optimal control models are presented where optimistic value and reaching index are chosen as the optimality criteria, respectively. Applying the proposed uncertain optimal control model to a portfolio selection problem, we obtain the uncertainty distribution of the first hitting time (the investors’ first profit time). Meanwhile, sufficient conditions of the optimal control strategy of such models are provided. Numerical simulations are provided which reveal the change for our optimal control strategy.

Hierarchical robust control for variable‐pitch wind turbine with actuator faults

AbstractIn this brief, we develop a fault tolerant control (FTC) using a hierarchical‐robust methodology with guaranteed stability for a wind turbine (WT), which is subject to an exogenous input and an actuator fault. The high‐level control is designed to robustly compensate for the nonlinearities, uncertainty, and disturbance in the system. The low‐level control is designed to robustly automatically allocate control authority among redundant actuators mitigating an actuator fault. At the low‐level, the robust control problem is transformed into an equivalent optimal control problem. Finally, the developed FTC is validated using a comprehensive simulation study on a 5 MW variable‐pitch WT model given in the Fatigue, Aerodynamics, Structures, and Turbulence (FAST) simulator provided by the U.S. National Renewable Energy Laboratory (NREL). The results show that the developed FTC retains robust performance in the presence of wind disturbance and a time‐varying actuator fault.

Robust mean field social control problems with applications in analysis of opinion dynamics

This paper investigates the social optimality of linear quadratic mean field control systems with unmodelled dynamics. The objective of agents is to optimise the social cost, which is the sum of costs of all agents. By variational analysis and direct decoupling methods, the social optimal control problem is analysed, and two equivalent auxiliary robust optimal control problems are obtained for a representative agent. By solving the auxiliary problem with consistent mean field approximations, a set of decentralised strategies is designed, and its asymptotic social optimality is further proved. Next, the results are applied into the study of opinion dynamics in social networks. The evolution of opinions is analysed over finite and infinite horizons, respectively. All opinions are shown to reach agreement with the average opinion in a probabilistic sense. Finally, local interactions among multiple sub-populations are examined via graphon theory.

Numerical solution of free final time fractional optimal control problems

The main purpose of this work is to develop a numerical solution method for solving a class of nonlinear free final time fractional optimal control problems. This problem is subject to equality and inequality constraints in canonical forms, and the orders in the fractional system can be different. For this problem, we first show that, by a time-scaling transformation, the problem can be transformed into an equivalent fractional optimal control problem with fixed final time. We then discretize the transformed fractional optimal control problem by a second-order one-point numerical integration scheme and the trapezoidal rule. Furthermore, we derive the gradient formulae of the cost and constraint functions with respect to decision variables and propose a numerical procedure for calculating these gradients. On this basis, a gradient-based optimization algorithm is developed for solving the resulting problem. Finally, numerical simulations of three example problems illustrate the effectiveness of the developed algorithm.

Fast Data-Driven Model Predictive Control Strategy for Connected and Automated Vehicles

Abstract We present a fast data-driven model predictive control (MPC) strategy for connected and automated vehicles, which can ensure collision avoidance in the presence of uncertainty in shared/predicted trajectory of preceding vehicles. The proposed control strategy focuses on improvement in fuel economy and computational efficiency. We exploit a data-driven modeling approach to identify a linear predictor for the nonlinear system and evaluate a deterministic equivalent of the probabilistic collision avoidance constraint to formulate the equivalent convex optimal control problem. We then develop a hierarchical control framework with sampling-based high-level control and fast MPC-based low-level control. Simulation results show the efficacy of the proposed approach both in terms of computation time and fuel efficiency.

Model predictive control for singular differential-algebraic equations

ABSTRACT We study model predictive control for singular differential-algebraic equations with higher index. This is a novelty when compared to the literature where only regular differential-algebraic equations with additional assumptions on the index and/or controllability are considered. By regularisation techniques, we are able to derive an equivalent optimal control problem for an ordinary differential equation to which well-known model predictive control techniques can be applied. This allows the construction of terminal constraints and costs such that the origin is asymptotically stable w.r.t. the resulting closed-loop system.

Numerical solution of arbitrary-order linear partial differential equations using an optimal control technique

This study aims to present a new approach for finding the numerical solution of linear partial differential equations using an optimal control technique. First, a partial differential equation is transformed into an equivalent optimal control problem and then, control and state variables are approximated by Chebychev series. Therefore, the obtained problem is converted to an optimization problem subject to algebraic equality constraints. Finally, a suitable iterative optimization technique is implemented to approximate the unknown coefficients of Chebychev polynomials to find the numerical solution of the original problem. In this method, a new idea is used, enabling us to deal with almost all kinds of linear partial differential equations with different types of initial and boundary conditions. Several kinds of numerical examples are solved to illustrate the accuracy and efficiency of the proposed method.

Transformation preserving controllability for nonlinear optimal control problems with joint boundary conditions

In this paper we develop a new approach for optimal control problems with general jointly varying state endpoints (also called coupled endpoints). We present a new transformation of a nonlinear optimal control problem with jointly varying state endpoints and pointwise equality control constraints into an equivalent optimal control problem of the same type but with separately varying state endpoints in double dimension. Our new transformation preserves among other properties the controllability (normality) of the considered optimal control problems. At the same time it is well suited even for the calculus of variations problems with joint state endpoints, as well as for optimal control problems with free initial and/or final time. This work is motivated by the results on the second order Sturm–Liouville eigenvalue problems with joint endpoints by Dwyer and Zettl (1994) and by the sensitivity result for nonlinear optimal control problems with separated state endpoints by the authors (2018).

Observer-based robust control for flexible-joint robot manipulators: A state-dependent Riccati equation-based approach

In this paper, the robust control problem is tackled by employing the state-dependent Riccati equation (SDRE) for uncertain systems with unmeasurable states subject to mismatched time-varying disturbances. The proposed observer-based robust (OBR) controller is applied to two highly nonlinear, coupled and large robotic systems: namely a manipulator presenting joint flexibility due to deformation of the power transmission elements between the actuator and the robot known as flexible-joint robot (FJR) and also an FJR incorporating geared permanent magnet DC motor dynamics in its dynamic model called electrical flexible-joint robot (EFJR). A novel state-dependent coefficient (SDC) form is introduced for uncertain EFJRs. Rather than coping with the OBR control problem for such complex uncertain robotic systems, the main idea is to solve an equivalent nonlinear optimal control problem where the uncertainty and disturbance bounds are incorporated in the performance index. The stability proof is presented. Solving the complicated robust control problem for FJRs and EFJRs subject to uncertainty and disturbances via a simple and flexible nonlinear optimal approach and no need of state measurement are the main advantages of the proposed control method. Finally, simulation results are included to verify the efficiency and superiority of the control scheme.

Optimal bounded control of stochastically excited strongly nonlinear vibro-impact system

An optimal bounded control strategy for strongly nonlinear vibro-impact systems under stochastic excitations with actuator saturation is proposed. First, the impact effect is incorporated in an equivalent equation by using a nonsmooth transformation. Under the assumption of light damping and weak random perturbation, the system energy is a slowly varying process. By using the stochastic averaging of envelope for strongly nonlinear systems, the partially averaged Itô stochastic differential equation for system energy can be derived. The optimal control problem is transformed from the original optimal control problem for the state variables to an equivalent optimal control problem for the system energy, which decreases the dimensions of the optimal control problem. Then, based on stochastic maximum principle, an adjoint equation for the adjoint variable and the maximum condition of partially averaged control problem are established. For infinite time-interval ergodic control, the adjoint variable is assumed to be a stationary process and the adjoint equation can be further simplified. Finally, the probability density function of the system energy and other statistics of the optimally controlled system are derived by calculating the associated Fokker–Plank–Kolmogorov equation. For comparison, the bang–bang control is also investigated and the control results are compared to show the advantages of the developed control strategy.

Adaptive dynamic programming based robust control of nonlinear systems with unmatched uncertainties

This paper proposes a new approach to address robust control design for nonlinear continuous-time systems with unmatched uncertainties. First, we transform the robust control problem into an equivalent optimal control problem, which allows to simplify the control design. A critic neural network (NN) is then adopted to reformulate the derived Hamilton–Jacobi–Bellman (HJB) equation based on the optimal control methodology. Then, a novel adaptation algorithm is used to online directly estimate the unknown NN weights, so as to achieve the guaranteed convergence. The control system stability and the convergence of the derived control action to the optimal solution can be rigorously proved. Finally, two simulation examples are provided to illustrate the validity and efficacy of the proposed method.

Numerical methods for construction of value functions in optimal control problems with infinite horizon

The article is devoted to the analysis of optimal control problems with infinite time horizon. These problems arise in economic growth models and in stabilization problems for dynamic systems. The problem peculiarity is a quality functional with an unbounded integrand which is discounted by an exponential index. The problem is reduced to an equivalent optimal control problem with the stationary value function. It is shown that the value function is the generalized minimax solution of the corresponding Hamilton-Jacobi equation. The boundary condition for the stationary value function is replaced by the property of the Hölder continuity and the sublinear growth condition. A backward procedure on infinite time horizon is proposed for construction of the value function. This procedure approximates the value function as the generalized minimax solution of the stationary Hamilton-Jacobi equation. Its convergence is based on the contraction mapping method defined on the family of uniformly bounded and Holder continuous functions. After the special change of variables the procedure is realized in numerical finite difference schemes on strongly invariant compact sets for optimal control problems and differential games.

An adaptive finite element method for the sparse optimal control of fractional diffusion

AbstractWe propose and analyze an a posteriori error estimator for a partial differential equation (PDE)‐constrained optimization problem involving a nondifferentiable cost functional, fractional diffusion, and control‐constraints. We realize fractional diffusion as the Dirichlet‐to‐Neumann map for a nonuniformly PDE and propose an equivalent optimal control problem with a local state equation. For such an equivalent problem, we design an a posteriori error estimator which can be defined as the sum of four contributions: two contributions related to the approximation of the state and adjoint equations and two contributions that account for the discretization of the control variable and its associated subgradient. The contributions related to the discretization of the state and adjoint equations rely on anisotropic error estimators in weighted Sobolev spaces. We prove that the proposed a posteriori error estimator is locally efficient and, under suitable assumptions, reliable. We design an adaptive scheme that yields, for the examples that we perform, optimal experimental rates of convergence.

Time-scaling transformation for optimal control problem with time-varying delay

This paper focuses on the solution to nonlinear time-delay optimal control problems with time-varying delay subject to canonical equality and inequality constraints. Traditional control parameterization in conjunction with time-scaling transformation could optimize control parameters and switching times at the same time when the time-delays in the dynamic system are taken as constants. The purpose of this paper is to extend this method to solve the dynamic systems with time-varying delay. We introduce a hybrid time-scaling transformation that converts the given time-delay system into an equivalent system defined on a new time horizon with fixed switching times. Meanwhile, we obtain the value of time-delay state utilizing the relationship between the new time scale and the original one. After computing the gradients of the cost and constraints with respect to the control heights and its durations, we could solve the equivalent optimal control problem using gradient based optimization method.

On Infinite Horizon Optimal Control Problems with a Feed Forward Neural Network Scheme

In this paper, a class of infinite-horizon nonlinear optimal control problems is considered. The main idea is to convert the infinite horizon problem to an equivalent finite-horizon optimal control problem. According to the Pontryagin minimum principle for optimal control problems and by constructing an error function, we define an unconstrained minimization problem. In the optimization problem, we use trial solutions for the state, costate and control functions where these trial solutions are constructed by using two-layer perceptron. We then minimize the error function where weights and biases associated with all neurons are unknown. Substituting the optimal values of the weights and biases into the trial solutions, we obtain the optimal solution of the original problem. We also use a dynamic optimization scheme to learning process and discuss the stability and convergence properties of it. Some examples are given to show the efficiency of the method.