Linear Quadratic Optimal Control Problem Research Articles

Value functional and optimal feedback control in linear-quadratic optimal control problem for fractional-order system

In this paper, a finite-horizon optimal control problem involving a dynamical system described by a linear Caputo fractional differential equation and a quadratic cost functional is considered. An explicit formula for the value functional is given, which includes a solution of a certain Fredholm integral equation. A step-by-step feedback control procedure for constructing $ \varepsilon $-optimal controls with any accuracy $ \varepsilon > 0 $ is proposed. The basis for obtaining these results is the study of a solution of the associated Hamilton–Jacobi–Bellman equation with so-called fractional coinvariant derivatives.

A new diagonalization based method for parallel-in-time solution of linear-quadratic optimal control problems

A new diagonalization technique for the parallel-in-time solution of linear-quadratic optimal control problems with time-invariant system matrices is introduced. The target problems are often derived from a semi-discretization of a Partial Differential Equation (PDE)-constrained optimization problem. The solution of large-scale time dependent optimal control problems is computationally challenging as the states, controls, and adjoints are coupled to each other throughout the whole time domain. This computational difficulty motivates the use of parallel-in-time methods. For time-periodic problems our diagonalization efficiently transforms the discretized optimality system into nt (=number of time steps) decoupled complex valued 2ny × 2ny systems, where ny is the dimension of the state space. These systems resemble optimality systems corresponding to a steady-state version of the optimal control problem and they can be solved in parallel across the time steps, but are complex valued. For optimal control problems with initial value state equations a direct solution via diagonalization is not possible, but an efficient preconditioner can be constructed from the corresponding time periodic optimal control problem. The preconditioner can be efficiently applied parallel-in-time using the diagonalization technique. The observed number of preconditioned GMRES iterations is small and insensitive to the size of the problem discretization.

Linear-quadratic optimal control problem for mean-field stochastic differential equations with a type of random coefficients

Linear-quadratic optimal control problem for mean-field stochastic differential equations with a type of random coefficients

Accurate solutions of interval linear quadratic regulator optimal control problems with fractional-order derivative

Accurate solutions of interval linear quadratic regulator optimal control problems with fractional-order derivative

Stochastic linear quadratic optimal control problem with terminal state constraints

AbstractThis paper addresses the stochastic linear quadratic (LQ) control problem with first‐ and second‐order moment constraints on the terminal state. The problem is a modified version of the optimal covariance control problem, where the terminal state is steered to a given probability distribution. Studying a multiplicative‐noise stochastic system rather than an additive‐noise system is a salient feature. Unlike the existing ideas in the optimal steering, by using the Lagrange multipliers method and establishing the stochastic maximum principle, our problem is converted into solving forward–backward stochastic difference equations (FBSDEs), which is a special stochastic two‐point boundary‐value problem (TPBVP). We provide the optimal closed‐loop controller and necessary and sufficient solvability conditions by developing a nonhomogeneous relationship between the optimal state and costate in FBSDEs. Finally, numerical examples are given to demonstrate our results.

Stochastic optimal control and piecewise parameterization and optimization method for inventory control system improvement

In recent decades, stochastic control systems have been widely used in industrial production, biomedicine, aerospace, military strategy and so on. In this paper, an approximate optimal strategy derived from a stochastic linear quadratic (SLQ) optimal control problem is considered, and a piecewise parameterization and optimization (PPAO) method is proposed. Firstly, using the principle of dynamic programming, the control form of SLQ optimal control problem is relevant to a Riccati differential equation. It is well known that the Riccati differential equation is difficult to be solved analytically. Thus, we present a PPAO method for finding an approximate optimal strategy for stochastic control problems. Here, a piecewise parameter control can be obtained by solving first-order differential equations rather than Riccati differential equations. Finally, the inventory control problems with different dimensions are used to justify the feasibility of PPAO method, and the results compared with the original optimal control are given. The results show that parametric control greatly simplifies the control form, and a small model error is ensured.

Linear-quadratic optimal control of second-order parabolic systems

This paper considers the linear-quadratic (LQ) optimal control problem for systems governed by a class of second-order parabolic partial differential equations (PDEs). This problem is significant in many areas of mathematics, control, engineering and so on, and thus has received great attention in the past decades. Different from the previous articles where the operator is applied to present the controller, the main contribution of this paper is to propose the discretisation-then-continuousization method, which is explicit and implementable. The solvability condition of the LQ optimal control problems is given based on a set of differential Riccati equations, and an explicit numerical calculation way of these equations and the design of the optimal controller are provided.

A generalization of the Riccati recursion for equality‐constrained linear quadratic optimal control

AbstractThis paper introduces a generalization of the well‐known Riccati recursion for solving the discrete‐time equality‐constrained linear quadratic optimal control problem. The recursion can be used to compute problem solutions as well as optimal feedback control policies. Unlike other tailored approaches for this problem class, the proposed method does not require restrictive regularity conditions on the problem. This allows its use in nonlinear optimal control problem solvers that use exact Lagrangian Hessian information. We demonstrate that our approach can be implemented in a highly efficient algorithm that scales linearly with the horizon length. Numerical tests show a significant speed‐up of about one order of magnitude with respect to state‐of‐the‐art general‐purpose sparse linear solvers. Based on the proposed approach, faster nonlinear optimal control problem solvers can be developed that are suitable for more complex applications or for implementations on low‐cost or low‐power computational platforms. The implementation of the proposed algorithm is made available as open‐source software.

Robust Solution of the Multi-Model Singular Linear-Quadratic Optimal Control Problem: Regularization Approach

We consider a finite horizon multi-model linear-quadratic optimal control problem. For this problem, we treat the case where the problem’s functional does not contain a control function. The latter means that the problem under consideration is a singular optimal control problem. To solve this problem, we associate it with a new optimal control problem for the same multi-model system. The functional in this new problem is the sum of the original functional and an integral of the square of the Euclidean norm of the vector-valued control with a small positive weighting coefficient. Thus, the new problem is regular. Moreover, it is a multi-model cheap control problem. Using the solvability conditions (Robust Maximum Principle), the solution of this cheap control problem is reduced to the solution of the following three problems: (i) a terminal-value problem for an extended matrix Riccati type differential equation; (ii) an initial-value problem for an extended vector linear differential equation; (iii) a nonlinear optimization (mathematical programming) problem. We analyze an asymptotic behavior of these problems. Using this asymptotic analysis, we design the minimizing sequence of state-feedback controls for the original multi-model singular optimal control problem, and obtain the infimum of the functional of this problem. We illustrate the theoretical results with an academic example.

Linear Quadratic Optimal Control of Discrete-Time Stochastic Systems Driven by Homogeneous Markov Processes

Random terms in many natural and social science systems have distinct Markovian characteristics, such as Markov jump-taking values in a finite or countable set, and Wiener process-taking values in a continuous set. In general, these systems can be seen as Markov-process-driven systems, which can be used to describe more complex phenomena. In this paper, a discrete-time stochastic linear system driven by a homogeneous Markov process is studied, and the corresponding linear quadratic (LQ) optimal control problem for this system is solved. Firstly, the relations between the well-posedness of LQ problems and some linear matrix inequality (LMI) conditions are established. Then, based on the equivalence between the solvability of the generalized difference Riccati equation (GDRE) and the LMI condition, it is proven that the solvability of the GDRE is sufficient and necessary for the well-posedness of the LQ problem. Moreover, the solvability of the GDRE and the feasibility of the LMI condition are established, and it is proven that the LQ problem is attainable through a certain feedback control when any of the four conditions is satisfied, and the optimal feedback control of the LQ problem is given using the properties of homogeneous Markov processes and the smoothness of the conditional expectation. Finally, a practical example is used to illustrate the validity of the theory.

The mean‐field linear quadratic optimal control problem for stochastic systems controlled by impulses

AbstractIn the present note, we state and solve the linear quadratic (LQ) control problem for a class of McKean–Vlasov stochastic differential equations for which the control actions are of impulsive nature. After reformulating the original optimization problem using an orthogonal decomposition of the state variables, we introduce an adequately defined system of two coupled matrix Lyapunov‐type differential equations with jumps. Such equations are central for the definition of the optimal feedback gains corresponding to the closed‐loop representation of the optimal LQ control.

Indefinite linear quadratic optimal control problem for uncertain random discrete-time systems

Indefinite linear quadratic optimal control problem for uncertain random discrete-time systems

Stochastic linear quadratic optimal control problems with expectation-type linear equality constraints on the terminal states

A stochastic linear quadratic (LQ) optimal control problem with an expectation-type linear equality constraint on the terminal state is considered. Under the solvability condition on a stochastic Riccati equation and a surjectivity condition on the linear constraint mapping, the constrained stochastic LQ problem is solved completely by the Lagrangian duality theory. Some equivalent characterizations of the surjectivity condition are discussed by the controllability theory of linear control systems. Especially, the equivalence between the surjectivity condition and a Kalman-type rank condition is proved under proper conditions.

Linear quadratic optimal control for time-delay stochastic system with partial information

ABSTRACT This paper is concerned with the linear quadratic optimal control problem for the time-delay stochastic system with partial information. The main contribution is to derive the equivalent solvability condition and give the explicitly optimal controller under partial information in terms of the Riccati equations. The key is to explicitly solve the forward and backward stochastic difference equations with partial information, which is derived from the stochastic maximum principle.

Stochastic Fokker–Planck Equations for Conditional McKean–Vlasov Jump Diffusions and Applications to Optimal Control

The purpose of this paper is to study optimal control of conditional McKean–Vlasov (mean-field) stochastic differential equations with jumps (conditional McKean–Vlasov jump diffusions, for short). To this end, we first prove a stochastic Fokker–Planck equation for the conditional law of the solution of such equations. Combining this equation with the original state equation, we obtain a Markovian system for the state and its conditional law. Furthermore, we apply this to formulate a Hamilton–Jacobi–Bellman equation for the optimal control of conditional McKean–Vlasov jump diffusions. Then we study the situation when the law is absolutely continuous with respect to Lebesgue measure. In that case the Fokker–Planck equation reduces to a stochastic partial differential equation for the Radon–Nikodym derivative of the conditional law. Finally we apply these results to solve explicitly the linear-quadratic optimal control problem of conditional stochastic McKean–Vlasov jump diffusions, and optimal consumption from a cash flow.

Linear quadratic optimal regulation for multiplicative noise systems with special terminal penalty

This paper focuses on the linear quadratic optimal control problem for stochastic systems with a special terminal penalty. The specialty lies in that the terminal penalty is a quadratic term over the conditional expectations of the terminal state. The essential difficulty lies in an additional measurability constraint on the input induced by the conditional expectation term, which is shared by the control problems for the stochastic system with delays, rational expectations problems, control problems involving conditional expectation terms, asymmetric information control and so on. A new idea, dynamic programming combined with orthogonal decomposition, is proposed to deal with this problem. The orthogonal decomposition eliminates the additional measurability constraint on the input. The solvability condition and the optimal controller are obtained by developing generalised Riccati equations. Numerical experiment is provided to support the achieved results.

Linear-quadratic optimal control problems of state delay systems under full and partial information

Linear-quadratic optimal control problems of state delay systems under full and partial information

The Global Maximum Principle for Progressive Optimal Control of Partially Observed Forward-Backward Stochastic Systems with Random Jumps

This paper is concerned with a partially observed progressive optimal control problem of forward-backward stochastic differential equations with random jumps, where the control domain is not necessarily convex, and the control variable enters into all the coefficients. In our model, the observation equation is not only driven by a Brownian motion but also a Poisson random measure, which also has correlated noises with the state equation. The partially observed global maximum principle is proved. To show its applications, a partially observed linear quadratic progressive optimal control problem of forward-backward stochastic differential equations with random jumps is investigated by the maximum principle and stochastic filtering. State estimate feedback representation of the optimal control is given in a more explicit form by introducing some ordinary differential equations.

Value Iteration for Continuous-Time Linear Time-Invariant Systems

Two data-driven strategies for value iteration in linear quadratic optimal control problems over an infinite horizon are proposed. The two architectures share common features, since they both consist of a purely continuous-time control architecture and are based on the forward integration of the Differential Riccati Equation (DRE). They profoundly differ, instead, in the estimation mechanism of the vector field of the underlying DRE from collected data: the first relies on a characterization of properties of the advantage function associated to the problem, whereas the second is inspired by tools from adaptive control theory and ensures semi-global exponential convergence to the optimal solution. Advantages and drawbacks of the architectures are discussed, while the performance is validated via a benchmark numerical example.

Sensitivity updates for linear-quadratic optimization problems in multi-step model predictive control

The paper discusses how parametric sensitivity analysis can be used in certain model predictive control (MPC) schemes. The sensitivity analysis will be performed with regard to the initial state measurement and update schemes will be derived that speed-up the computations. Throughout we restrict the discussion to linear-quadratic optimal control problems in discrete time, which frequently arise in tracking tasks with MPC. The derived tools from sensitivity analysis can be embedded into MPC schemes with a prediction step and multi-step MPC schemes with re-optimization and prediction step. Numerical experiments illustrating the sensitivity analysis are presented.