Quadratic Optimal Control Problem Research Articles

Linear quadratic optimal control of stochastic 2-D Roesser models

This paper investigates the linear quadratic (LQ) optimal control problem for the stochastic two-dimensional (2-D) systems governed by Roesser models with multiplicative noise. The main contribution is to give the necessary and sufficient optimality condition by proposing a set of novel forward and backward stochastic partial difference equations (FBSPDE), and to further present the explicitly optimal feedback control laws on the finite horizon and on the infinite horizon based on the Riccati-like difference equations and the algebraic equation, respectively. Several numerical simulations are provided to illustrate the performance of the designed controllers.

System transformation and model-free value iteration algorithms for continuous-time linear quadratic stochastic optimal control problems

In this paper, we investigate a continuous-time linear quadratic stochastic optimal control (LQSOC) problem in an infinite horizon, where diffusion and drift terms of the corresponding stochastic system depend on both state and control variables. In light of the stochastic control theory, this LQSOC problem is reduced to solving a generalised algebraic Riccati equation (GARE). With the help of an existing model-based value iteration (VI) algorithm, we propose two data-driven VI algorithms to solve the GARE. The first one relies on transforming the stochastic system into a deterministic control system first and then solving the LQSOC problem by the data of the deterministic system. Consequently, this algorithm does not need the information of two system coefficients and has a lower algorithm complexity. The second algorithm directly uses the data generated by the stochastic system, and thus it circumvents the requirement of all system coefficients. We also provide convergence proofs of these two data-driven algorithms and validate these algorithms through two simulation examples.

Finite-horizon and infinite-horizon linear quadratic optimal control problems: A data-driven Euler scheme

In this paper, we devote our focus to studying finite-horizon and infinite-horizon linear quadratic optimal control (LQOC) problems in continuous time. We focus on algorithms for approximating optimal controls of these two problems without the knowledge of all system coefficients. Firstly, we transform the backward differential Riccati equation (DRE) corresponding to the finite-horizon LQOC problem into a forward ordinary differential equation. Then, we discretize the forward equation by the Euler method. Subsequently, we use the collected state and input data to calculate all discretized points of the forward equation. At the same time, we obtain the optimal control of the finite-horizon LQOC problem. Moreover, we approximate the optimal control of the infinite-horizon LQOC problem by virtue of an asymptotic behavior between the DRE and an algebraic Riccati equation (ARE) arising in the infinite-horizon LQOC problem. Finally, we validate the obtained results via two practically motivated examples.

Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the Related Hamiltonian Systems

Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the Related Hamiltonian Systems

Linear quadratic optimal control problems of infinite‐dimensional mean‐field type with jumps

AbstractIn this paper, we formulate and investigate a framework for the theory of the linear quadratic optimal control problem (LQ problem) for infinite‐dimensional mean‐field stochastic evolution systems with jumps. We ensure the well‐posedness of the investigated problems by establishing the existence, uniqueness, and a priori estimates for mild solutions to general infinite‐dimensional mean‐field forward stochastic evolution equations (MF‐SEE) and mean‐field backward stochastic evolution equations (MF‐BSEE) with jumps. Leveraging the Yosida approximation theory, we establish a dual theory between MF‐SEE and MF‐BSEE with jumps, overcoming the challenge posed by the inapplicability of Itô's formula in the context of mild solutions. Our main results regarding the existence and uniqueness of the optimal control, along with corresponding dual characterizations and state feedback representations, are obtained through convex analysis techniques, our established dual theory, and decoupling methods.

Linear quadratic optimal control turnpike in finite and infinite dimension: Two-term expansion of the value function

In this paper, we consider a linear quadratic (LQ) optimal control problem in both finite and infinite dimensions. We derive an asymptotic expansion of the value function as the fixed time horizon T tends to infinity. The leading term in this expansion, proportional to T, corresponds to the optimal value attained through the classical turnpike theory in the associated static problem. The remaining terms are associated with optimal stabilization problems towards the turnpike.

Optimal control for both forward and backward discrete-time systems

Forward discrete-time systems use past information to update the current state, while backward discrete-time systems use future information to update the current state. This study focuses on optimal control problems within the context of forward and backward discrete-time systems. We begin by investigating a general optimal control problem for both forward and backward discrete-time systems. Leveraging the inherent properties of these systems and the Bellman optimality principle, we derive recursive equations as a means to solve such optimal control problems. Using these recursive equations, we obtain analytical expressions for both the optimal controls and optimal values of bang–bang and linear quadratic optimal control problems. Finally, we present a numerical example and an industrial wastewater treatment problem to illustrate and demonstrate our findings.

Pointwise Error Estimates of Numerical Solutions to Linear Quadratic Optimal Control Problems

An auxiliary optimal control problem is formulated that provides with its unique solution, a continuous representation of the global error of a numerical approximation to the solution of a linear quadratic optimal control problem. The resulting error functions are characterized as the unique solutions to an optimality system that is reformulated as a boundary value problem. With this formulation, reliable pointwise error estimates can be generated utilizing well-established techniques of defect control. A novel algorithm based on defect correction and defect control is presented that generates pointwise approximations to the global error of numerical optimal control solutions on a uniform grid. It is proven and numerically validated that this algorithm can generate pointwise estimates that approximate the true global error with a prescribed accuracy.

The Meaning and Accuracy of the Improving Functions in the Solution of the CBQR by Krotov’s Method

A new solution to the continuous-time bilinear quadratic regulator optimal control problem (CBQR) was recently developed using Krotov’s Method. This paper provides two theoretical results related to the properties of that solution. The first discusses the equivalent representation of the cost-to-go performance index. The second one breaks down this equivalence into smaller identities referencing the components of the performance index. The paper shows how these results can be used to verify the numerical accuracy of the computed solution. Additionally, the meaning of the improving function and the equivalent representation, which are the main elements in the discussed CBQR’s solution, are explained according to the derived notions. A numerical example of structural control application exemplifies the significance of these results and how they can be applied to a specific CBQR problem.

State‐dependent indirect hp‐pseudospectral method for rigid spacecraft attitude optimal control problem

AbstractThe state‐dependent indirect ‐pseudospectral (SDIHP) method for rigid spacecraft attitude control is presented in this paper. The optimal controller is obtained by the SDIHP method through classifying the attitude optimal control problem as a nonlinear quadratic optimal control problem. In the proposed control scheme, successive state‐dependent coefficient (SDC) parameterization is initially applied to the spacecraft dynamics to derive an available linear system to decrease the calculation cost and speed. Then, the indirect ‐adaptive spectral discretization is developed to convert the resultant optimal control problem into a sequence of linear equations, which can improve the global computational efficiency and guarantee the solving precision. Hence, the optimal control can be efficiently solved using basic matrix operations under the necessary collocation points based on the ‐adaptive features. Considering the inertia uncertainties and external disturbances, a state feedback controller is generated by the receding horizon control (RHC) framework to improve the robustness. Simulation results are included to show the effectiveness and performance of the suggested optimal method.

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

Linear quadratic stochastic optimal control with state- and control-dependent noises: A deterministic data approach

This paper investigates an infinite-horizon linear quadratic stochastic optimal control (LQSOC) problem with state- and control-dependent noises, in which two system matrices are unknown. First, a deterministic system is introduced and a relationship among its system trajectory, its control and the matrices to be solved is formulated. Subsequently, based on the deterministic system, an adaptive dynamic programming (ADP) algorithm is established to tackle the LQSOC problem. Then, the convergence analysis is presented by proving the equivalence between the proposed algorithm and an existing policy iteration algorithm. Finally, the effectiveness of the obtained algorithm is verified by a perturbed turbocharged diesel engine optimal control design example.

Lagrangian dual method for solving stochastic linear quadratic optimal control problems with terminal state constraints

A stochastic linear quadratic (LQ) optimal control problem with a pointwise linear equality constraint on the terminal state is considered. A strong Lagrangian duality theorem is proved under a uniform convexity condition on the cost functional and a surjectivity condition on the linear constraint mapping. Based on the Lagrangian duality, two approaches are proposed to solve the constrained stochastic LQ problem. First, a theoretical method is given to construct the closed-form solution by the strong duality. Second, an iterative algorithm, called augmented Lagrangian method (ALM), is proposed. The strong convergence of the iterative sequence generated by ALM is proved. In addition, some sufficient conditions for the surjectivity of the linear constraint mapping are obtained.

Linear quadratic optimal control problems for stochastic evolution equations in infinite horizon

Linear quadratic optimal control problems for stochastic evolution equations in infinite horizon

Time-inconsistent linear quadratic optimal control problem for forward–backward stochastic differential equations

We study the time-inconsistent linear quadratic optimal control problem for forward–backward stochastic differential equations with potentially indefinite cost weighting matrices for both the state and the control variables. Our research makes two contributions. Firstly, we introduce a novel type of Riccati equation system with parameters and constraint conditions, known as the generalized equilibrium Riccati equation. This equation system offers a comprehensive solution for the closedloop equilibrium strategy of the problem at hand. Secondly, we establish the well-posedness of the generalized equilibrium Riccati equation for the one-dimensional case, provided certain conditions are met.

Accurate solutions of interval linear quadratic regulator optimal control problems with fractional-order derivative

Accurate solutions of interval linear quadratic regulator optimal control problems with fractional-order derivative

Stochastic linear quadratic optimal control problem with terminal state constraints

AbstractThis paper addresses the stochastic linear quadratic (LQ) control problem with first‐ and second‐order moment constraints on the terminal state. The problem is a modified version of the optimal covariance control problem, where the terminal state is steered to a given probability distribution. Studying a multiplicative‐noise stochastic system rather than an additive‐noise system is a salient feature. Unlike the existing ideas in the optimal steering, by using the Lagrange multipliers method and establishing the stochastic maximum principle, our problem is converted into solving forward–backward stochastic difference equations (FBSDEs), which is a special stochastic two‐point boundary‐value problem (TPBVP). We provide the optimal closed‐loop controller and necessary and sufficient solvability conditions by developing a nonhomogeneous relationship between the optimal state and costate in FBSDEs. Finally, numerical examples are given to demonstrate our results.

Stochastic optimal control and piecewise parameterization and optimization method for inventory control system improvement

In recent decades, stochastic control systems have been widely used in industrial production, biomedicine, aerospace, military strategy and so on. In this paper, an approximate optimal strategy derived from a stochastic linear quadratic (SLQ) optimal control problem is considered, and a piecewise parameterization and optimization (PPAO) method is proposed. Firstly, using the principle of dynamic programming, the control form of SLQ optimal control problem is relevant to a Riccati differential equation. It is well known that the Riccati differential equation is difficult to be solved analytically. Thus, we present a PPAO method for finding an approximate optimal strategy for stochastic control problems. Here, a piecewise parameter control can be obtained by solving first-order differential equations rather than Riccati differential equations. Finally, the inventory control problems with different dimensions are used to justify the feasibility of PPAO method, and the results compared with the original optimal control are given. The results show that parametric control greatly simplifies the control form, and a small model error is ensured.

A generalization of the Riccati recursion for equality‐constrained linear quadratic optimal control

AbstractThis paper introduces a generalization of the well‐known Riccati recursion for solving the discrete‐time equality‐constrained linear quadratic optimal control problem. The recursion can be used to compute problem solutions as well as optimal feedback control policies. Unlike other tailored approaches for this problem class, the proposed method does not require restrictive regularity conditions on the problem. This allows its use in nonlinear optimal control problem solvers that use exact Lagrangian Hessian information. We demonstrate that our approach can be implemented in a highly efficient algorithm that scales linearly with the horizon length. Numerical tests show a significant speed‐up of about one order of magnitude with respect to state‐of‐the‐art general‐purpose sparse linear solvers. Based on the proposed approach, faster nonlinear optimal control problem solvers can be developed that are suitable for more complex applications or for implementations on low‐cost or low‐power computational platforms. The implementation of the proposed algorithm is made available as open‐source software.

A numerical method for solving quadratic fractional optimal control problems

The objective of this article is to present a novel algorithm that can efficiently address fractional quadratic optimal control problems (FQOCPs) through the application of the generalized differential transform method, in conjunction with a Vandermonde matrix. The algorithm’s performance, in terms of solution accuracy, reliability, and efficiency, is exemplified by a range of illustrative examples. This paper introduces an innovative methodology for the numerical resolution of FQOCPs, demonstrating its inherent capabilities and efficacy.