Optimal Feedback Control Problem Research Articles

Uncertainty-resilient constrained rendezvous trajectory optimization via stochastic feedback control and unscented transformation

This paper proposes an innovative approach for constrained rendezvous trajectory optimization (CRTO) using stochastic optimal feedback control and unscented transform (UT) uncertainty quantification. The method overcomes limitations of traditional deterministic CRTO solutions that suffer from uncontrolled terminal state error growth and severely deteriorated path constraint satisfaction when initial state uncertainty, dynamics uncertainty, and navigation uncertainty are considered. The approach involves constructing a stochastic optimal feedback control (SOFC) problem with chance constraints and introducing linear feedback control to regulate both the mean and variance of terminal state errors. UT is employed to approximately quantify the state errors and their propagation, transforming the SOFC problem into an unconstrained deterministic optimization (UDO) problem. Differential dynamic programming (DDP) is then used to solve the UDO problem. The obtained stochastic optimal solution provides a robust rendezvous trajectory and a corresponding explicit closed-loop guidance law, improving terminal accuracy and path constraint satisfaction in the presence of various considered uncertainties. The influence of different term weights in the objective function on terminal accuracy and path constraint satisfaction is also studied. The effectiveness of the proposed approach is verified through Monte Carlo simulation, demonstrating the robustness of the closed-loop control strategy. The results highlight the potential of the method in enhancing terminal accuracy and path constraint satisfaction in uncertain rendezvous scenarios.

An Economic Model Predictive Control for Knowledge Transmission Processes in Multilayer Complex Networks.

In this article, we study the optimal feedback control problems of knowledge dissemination processes in multilayer complex networks. First, a node-based model is established in multilayer complex networks and two collaborative control strategies are exerted to increase the scope and speed of knowledge dissemination, forming a closed-loop control system. Then, we develop a two-layer optimal control framework. At the upper level, the optimal solution of the control system is solved and sent to the lower layer. At the lower level, a model predictive controller (MPC) receives input information from the upper level and is formulated to decide on the network and then transmits it to its heterogeneous networks which can reduce control resources and computation complexity. Finally, numerical simulations are conducted to confirm the theoretical results.

Density estimation based soft actor-critic: deep reinforcement learning for static output feedback control with measurement noise

ABSTRACT The state-of-the-art deep reinforcement learning (DRL) methods, including Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), among others, demonstrate significant capability in solving the optimal static state feedback control (SSFC) problem. This problem can be modeled as a fully observed Markov decision process (MDP). However, the optimal static output feedback control (SOFC) problem with measurement noise is a typical partially observable MDP (POMDP), which is difficult to solve, especially for the continuous state-action-observation space with high dimensions. This paper proposes a two-stage framework to address this challenge. In the laboratory stage, both the states and the noisy outputs are observable; the SOFC policy is converted to a constrained stochastic SSFC policy, of which the probability density function is generally not analytical. To this end, a density estimation based SAC algorithm is proposed to explore the optimal SOFC policy by learning the optimal constrained stochastic SSFC. Consequently, in the real-world stage, only the noisy outputs and the learned SOFC policy are required to solve the optimal SOFC problem. Numerical simulations and the corresponding experiments with robotic arms are provided to illustrate the effectiveness of our method. The code is available at https://github.com/RanKyoto/DE-SAC.

Direct integral pseudospectral and integral spectral methods for solving a class of infinite horizon optimal output feedback control problems using rational and exponential Gegenbauer polynomials

This study is concerned with the numerical solution of a class of infinite-horizon linear regulation problems with state equality constraints and output feedback control. We propose two numerical methods to convert the optimal control problem into nonlinear programming problems (NLPs) using collocations in a semi-infinite domain based on rational Gegenbauer (RG) and exponential Gegenbauer (EG) basis functions. We introduce new properties of these basis functions and derive their quadratures and associated truncation errors. A rigorous stability analysis of the RG and EG interpolations is also presented. The effects of various parameters on the accuracy and efficiency of the proposed methods are investigated. The performance of the developed integral spectral method is demonstrated using two benchmark test problems related to a simple model of a divert control system and the lateral dynamics of an F-16 aircraft. Comparisons of the results of the current study with available numerical solutions show that the developed numerical scheme is efficient and exhibits faster convergence rates and higher accuracy.

On the Rod Heating Problem by Sources on the Class of Zonal Controls Using the Current and Past State at Measurement Points

We study an optimal feedback control problem for the rod heating process by means of lumped sources. The control actions are the powers of the sources, the values of which are defined on the class of zonal controls. The values of the parameters of zonal control actions are determined by subsets of the state space, to which belong the values of the process state at the measurement points at the current and past time moments. The posed problem is reduced to a parametric optimal control problem on determining a finite-dimensional vector of values of the parameters of zonal control actions. We have obtained optimality conditions for the values of the parameters of zonal control actions. These conditions contain formulas for the gradient of the objective functional with respect to the optimizable parameters. They make it possible to solve the reduced problem numerically with the application of efficient first-order optimization methods.

Https://www.bakumathj.org/archive/Vol2No2/j.bmj.033.pdf

In the paper, we consider an approach for numerical solution to the optimal feedback control problem for an object with distributed parameters on the basis of observation of the object’s phase state at its specific locations by the example of the rod heating process. The control actions are the power of the heat source, the values of which are defined on the class of “zonal” controls. The values of the parameters of zonal control actions are determined by subsets of the state space, to which belong the values of the process state at the measurement points at the current moment of time. The problem of determining zonal controls is reduced to a parametric optimal control problem on determining a finite-dimensional vector of values of the parameters of zonal control actions. We derive optimality conditions for the values of the parameters of zonal control actions. These conditions contain formulas for the gradient of the objective functional with respect to the optimizable parameters of zonal controls. They make it possible to solve the stated problem numerically with the application of efficient first-order optimization methods.

Optimal control problems for the reaction–diffusion–convection equation with variable coefficients

The solvability of optimal control problems is proved on both weak and strong solutions of a boundary value problem for the nonlinear reaction–diffusion–convection equation with variable coefficients. In the second case, the requirements for smoothness of the multiplicative control are reduced. The study of extremal problems is based on the proof of the solvability of the corresponding boundary value problems and on the qualitative analysis of their solutions properties. The large data existence results for weak solutions, the maximum principle as well as the local existence and uniqueness of a strong solution are established. Moreover, an optimal feedback control problem is considered. Using methods of the theory of topological degree for set-valued perturbations (with aspheric image sets) of generalized monotone operators, we obtain sufficient conditions for the solvability of this problem in the class of weak solutions.

Computational method for feedback perimeter control of multiregion urban traffic networks with state-dependent delays

Perimeter control is the manipulation of traffic flow by adjusting traffic signals on the borders between regions. It can be used to address the problem of traffic congestion in urban areas. Although there are many papers devoted to perimeter control, the dependence of time delay on traffic state is rarely considered. Moreover, most of them depend on accumulation reference points of the network and the stability of given equilibrium points to derive a perimeter controller. In reality, such equilibrium points and the optimal reference points are unavailable in advance. With these in minds, this paper formulates an optimal feedback perimeter control problem (OFPCP) subject to state-dependent delays without given information about the optimal reference point and equilibrium point. Different from the control method based on linearization or partial linearization, a hybrid algorithm combining filled function method and gradient-based method is introduced to solve this challenging optimal feedback control problem. Some experiments are performed to demonstrate the effectiveness of our method.

Finite-Time Stabilization and Optimal Feedback Control for Nonlinear Discrete-Time Systems

Finite-time stability involves dynamical systems whose trajectories converge to an equilibrium state in finite time. Sufficient conditions for finite-time stability have recently been developed in the literature for discrete-time dynamical systems. In this article, we build on these results to develop a framework for addressing the problem of optimal nonlinear analysis and feedback control for finite-time stability and finite-time stabilization for nonlinear discrete-time controlled dynamical systems. Finite-time stability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that satisfies a difference inequality involving fractional powers and a minimum operator. This Lyapunov function can clearly be seen to be the solution to a difference equation that corresponds to a steady-state form of the Bellman equation, and hence, guaranteeing both finite-time stability and optimality. Finally, a numerical example is presented to demonstrate the efficacy of the proposed finite-time discrete stabilization framework.

Optimal feedback control problems for a semi-linear neutral retarded integro-differential system

This article considers the optimal and time optimal feedback control problems for a semi-linear neutral retarded integro-differential system. The existence of mild solutions and feasible pairs for the considered system is studied by applying theory of resolvent operators for linear neutral integro-differential evolution systems, fractional powers of operators and Schauder fixed point theorem. Then the Lagrange optimal feedback control problem for the system is investigated via limit arguments under some suitable assumptions. The time optimal feedback control problem is proposed and discussed deliberately here as well. An example is presented in the end to illustrate the obtained results.

Machine Learning Feedback Control Approach Based on Symbolic Regression for Robotic Systems

A control system of an autonomous robot produces a control signal based on feedback. This type of control implies the control of an object according to its state that is mathematically the control synthesis problem. Today there are no universal analytical methods for solving the general synthesis problem, and it is solved by certain particular approaches depending on the type of control object. In this paper, we propose a universal numerical approach to solving the problem of optimal control with feedback using machine learning methods based on symbolic regression. The approach is universal and can be applied to various objects. However, the use of machine learning methods imposes two aspects. First, when using them, it is necessary to reduce the requirements for optimality. In machine learning, optimization algorithms are used, but strictly optimal solutions are not sought. Secondly, in machine learning, analytical proofs of the received properties of solutions are not required. In machine methods, a set of tests is carried out and it is shown that this is sufficient to achieve the required properties. Thus, in this article, we initially introduce the fundamentals of machine learning control, introduce the basic concepts, properties and machine criteria for application of this technique. Then, with regard to the introduced notations, the feedback optimal control problem is considered and reformulated in order to add to the problem statement that such a property adjusts both the requirements of stability and optimality. Next, a description of the proposed approach is presented, theoretical formulations are given, and its efficiency is demonstrated on the computational examples in mobile robot control tasks.

Finite-time optimal feedback control mechanism for knowledge transmission in complex networks via model predictive control

In this paper, the finite-time (FT) optimal feedback control problems of knowledge transmission processes in complex networks via model predictive control (MPC) have been studied. Firstly, we build a knowledge transmission Susceptible–Infected–Hesitation (SIH) model in complex networks. Secondly, interventional control strategies are designed to regulate the system parameters to improve the performance of knowledge dissemination, including improving self-learning ability, acquaintance influence, and review rate. With the help of the Lyapunov-based HJB optimal control method, the existence of the optimal solution to the economic optimal problem of the knowledge transmission control model is guaranteed. Then, the optimal control solution is derived by using Pontryagin’s maximum principle. To focus on the performance indicators and state trajectories of the system and enable the controller to modify in real-time according to the state in a fixed time interval as soon as possible, an MPC based on FT feedback is proposed for the first time. Furthermore, under feedback control and initial conditions, the control knowledge dissemination model is FT stable. Numerical simulations are provided to verify the proposed method.

Optimal feedback control for a class of fractional evolution equations with history-dependent operators

In this paper, we will study optimal feedback control problems derived by a class of Riemann-Liouville fractional evolution equations with history-dependent operators in separable reflexive Banach spaces. We firstly introduce suitable hypotheses to prove the existence and uniqueness of mild solutions for this kind of Riemann-Liouville fractional evolution equations with history-dependent operators. Then, by introducing a feedback iterative technique and applying Filippov theorem, we show the existence of feasible pairs and optimal control pairs of the optimal feedback control systems with history-dependent operators. Finally, we give some applications to illustrate our main results.

Solvability of the Non-Linearly Viscous Polymer Solutions Motion Model.

In this paper we consider the initial–boundary value problem describing the motion of weakly concentrated aqueous polymer solutions. The model involves the regularized Jaumann’s derivative in the rheological relation. Also this model is considered with non-linear viscosity. On the basis of the topological approximation approach to the study of hydrodynamics problems the existence of weak solutions is proved. Also we consider an optimal feedback control problem for this initial–boundary value problem. The existence of an optimal solution minimizing a given performance functional is proved.

Approximative Policy Iteration for Exit Time Feedback Control Problems Driven by Stochastic Differential Equations using Tensor Train Format

We consider a stochastic optimal exit time feedback control problem. The Bellman equation is solved approximatively via the Policy Iteration algorithm on a polynomial ansatz space by a sequence of linear equations. As high degree multi-polynomials are needed, the corresponding equations suffer from the curse of dimensionality even in moderate dimensions. We employ tensor-train methods to account for this problem. The approximation process within the Policy Iteration is done via a Least-Squares ansatz and the integration is done via Monte-Carlo methods. Numerical evidences are given for the (multi dimensional) double well potential and a three-hole potential.

Optimal feedback control for a class of fed-batch fermentation processes using switched dynamical system approach

<abstract><p>This paper considers an optimal feedback control problem for a class of fed-batch fermentation processes. Our main contributions are as follows. Firstly, a dynamic optimization problem for fed-batch fermentation processes is modeled as an optimal control problem of switched dynamical systems, and a general state-feedback controller is designed for this dynamic optimization problem. Unlike the existing switched dynamical system optimal control problem, the state-dependent switching method is applied to design the switching rule, and the structure of this state-feedback controller is not restricted to a particular form. Then, this problem is transformed into a mixed-integer optimal control problem by introducing a discrete-valued function. Furthermore, each of these discrete variables is represented by using a set of 0-1 variables. By using a quadratic constraint, these 0-1 variables are relaxed such that they are continuous on the closed interval $ [0, 1] $. Accordingly, the original mixed-integer optimal control problem is transformed intoa nonlinear parameter optimization problem. Unlike the existing works, the constraint introduced for these 0-1 variables are at most quadratic. Thus, it does not increase the number of locally optimal solutions of the original problem. Next, an improved gradient-based algorithm is developed based on a novel search approach, and a large number of numerical experiments show that this novel search approach can effectively improve the convergence speed of this algorithm, when an iteration is trapped to a curved narrow valley bottom of the objective function. Finally, numerical results illustrate the effectiveness of this method developed by this paper.</p></abstract>

Balanced Energy/Time Optimal Defibrillation

Abstract The synthesis of a cardiac defibrillating pulse is cast as a standard optimal feedback control problem that minimizes a weighted measure of consumed energy and elapsed time to reach a nominal defibrillated state. The solution is developed for a general first-order system that includes the widely used parallel resistor/capacitor circuit and energy source as a special case. The novel optimal pulse comprises an exponentially ascending and a rectangular component; it is agile and energy conscious; and it therefore outperforms the waveforms developed thus far that minimize energy or time expenditures alone. Explicit time-domain expressions are derived which may be used for comparing against other commonly studied defibrillating functions. The analytic formulas and computer simulations may be useful to implement performance improvements in defibrillating devices.

Output-Feedback Based Simplified Optimized Backstepping Control for Strict-Feedback Systems with Input and State Constraints

In this paper, an adaptive neural-network (NN) output feedback optimal control problem is studied for a class of strict-feedback nonlinear systems with unknown internal dynamics, input saturation and state constraints. Neural networks are used to approximate unknown internal dynamics and an adaptive NN state observer is developed to estimate immeasurable states. Under the framework of the backstepping design, by employing the actor-critic architecture and constructing the tan-type Barrier Lyapunov function (BLF), the virtual and actual optimal controllers are developed. In order to accomplish optimal control effectively, a simplified reinforcement learning (RL) algorithm is designed by deriving the updating laws from the negative gradient of a simple positive function, instead of employing existing optimal control methods. In addition, to ensure that all the signals in the closed-loop system are bounded and the output can follow the reference signal within a bounded error, all state variables are confined within their compact sets all times. Finally, a simulation example is given to illustrate the effectiveness of the proposed control strategy.

Control for networked control systems with multiplicative noises, packet dropouts and multiple delays

This paper mainly focuses on the optimal output feedback control problem for networked control systems (NCSs) involving multiplicative noises, packet dropouts, input delays and measurement delays. The main contributions of this paper can be concluded as follows. Firstly, different from the previous results, this paper overcomes the barrier of the packet dropouts and measurement delays. Based on the measurement data, the optimal estimator is given. Secondly, by using maximum principle, a sufficient and necessary condition for the optimal control problem is presented. Moreover, the explicit output feedback controller is derived with feedback gain based on the coupled Riccati difference equations. Numerical example is illustrated to show the effectiveness of the proposed algorithm.

Feedback Control Problem for Modified Kelvin-Voigt Model

This paper deals with the optimal feedback control problem for the modified Kelvin-Voigt model. The considered model describes the motion of weakly concentrated aqueous polymer solutions. In our case, the control function (the external force) depends on the velocity of the fluid. In such a way, the control is not selected from a finite set of available controls, but belongs to the image of some multi-valued map. The solution for the control problem of fluid motion is a pair: the velocity of the fluid and the control (the density of external forces). Since there can be many such pairs, the concept of optimal solution naturally arises, which gives a minimum to specified cost functional. For the considered optimal feedback control problem the existence theorem on weak solution is proved.