Linear Quadratic Control Problem Research Articles

Modeling and computation of an integral operator Riccati equation for an infinite-dimensional stochastic differential equation governing streamflow discharge

We propose a linear-quadratic (LQ) control problem of streamflow discharge by optimizing an infinite-dimensional jump-driven stochastic differential equation (SDE). Our SDE is a superposition of Ornstein-Uhlenbeck processes (supOU process), generating a sub-exponential autocorrelation function observed in actual data. The integral operator Riccati equation is heuristically derived to determine the optimal control of the infinite-dimensional system. In addition, its finite-dimensional version is derived with a discretized distribution of the reversion speed and computed by a finite difference scheme. The optimality of the Riccati equation is analyzed by a verification argument. The supOU process is parameterized based on the actual data of a perennial river. The convergence of the numerical scheme is analyzed through computational experiments. Finally, we demonstrate the application of the proposed model to realistic problems along with the Kolmogorov backward equation for the performance evaluation of controls.

Backward Stochastic Volterra Integro-Differential Equations and Applications in Optimal Control Problems

In this article, a class of backward stochastic Volterra integro-differential equations (BSVIDEs) is introduced and studied. It is worthy mentioning that the proposed BSVIDEs cannot be covered by the existing backward stochastic Volterra integral equations (BSVIEs), and they also have the nice flow property such that Itô's formula becomes quite applicable. It is found that BSVIDEs can provide a neat sufficient condition for the solvability of BSVIEs with generator depending on the diagonal value of the solutions. As applications, the optimal control problems in terms of maximum principles and linear quadratic control problems of optimal control for forward stochastic Volterra integro-differential equations are investigated. In contrast with the BSVIEs in current literature, some interesting phenomena and advantages of BSVIDEs are revealed. (A corrected version is attached.)

Optimal Reinsurance and Investment Strategies Under Mean-Variance Criteria: Partial and Full Information

This paper is concerned with an optimal reinsurance and investment problem for an insurance firm under the criterion of mean-variance. The driving Brownian motion and the rate in return of the risky asset price dynamic equation cannot be directly observed. And the short-selling of stocks is prohibited. The problem is formulated as a stochastic linear-quadratic control problem where the control variables are constrained. Based on the separation principle and stochastic filtering theory, the partial information problem is solved. Efficient strategies and efficient frontier are presented in closed forms via solutions to two extended stochastic Riccati equations. As a comparison, the efficient strategies and efficient frontier are given by the viscosity solution to the HJB equation in the full information case. Some numerical illustrations are also provided.

Fixed‐time linear quadratic adaptive sliding mode control for a class of cart‐pendulum robots

This article is concerned with the fixed-time path-following control problem of a perturbed cart-pendulum robot with linear quadratic (LQ) performance optimization. In order to ensure the displacements of cart and pendulum of the robot following the desired paths within fixed-time, adaptive anti-perturbation sliding mode control schemes with a novel fixed-time sliding mode surface are developed to withstand the adverse influence of perturbations. Furthermore, an adaptive radial basis function neural network (RBFNN)-based sliding mode control algorithm is employed to approximate the continuous perturbations, so that the path-following LQ control problem can be further dealt with. The fixed-time stability and path-following results of the cart-pendulum robotic system are achieved by Lyapunov stability theory. A simulation example for a heavy material handling agricultural robot is proposed to verify the effectiveness of the developed method.

Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions

We study the problem of learning-augmented predictive linear quadratic control. Our goal is to design a controller that balances "consistency'', which measures the competitive ratio when predictions are accurate, and "robustness'', which bounds the competitive ratio when predictions are inaccurate. We propose a novel λ-confident controller and prove that it maintains a competitive ratio upper bound of 1 + min {O(λ2ε)+ O(1-λ)2,O(1)+O(λ2)} where λ∈ [0,1] is a trust parameter set based on the confidence in the predictions, and ε is the prediction error. Further, motivated by online learning methods, we design a self-tuning policy that adaptively learns the trust parameter λ with a competitive ratio that depends on ε and the variation of system perturbations and predictions. We show that its competitive ratio is bounded from above by 1+O(ε) /(Θ)(1)+Θ(ε))+O(μVar) where μVar measures the variation of perturbations and predictions. It implies that by automatically adjusting the trust parameter online, the self-tuning scheme ensures a competitive ratio that does not scale up with the prediction error ε.

DeepSets and Their Derivative Networks for Solving Symmetric PDEs

Machine learning methods for solving nonlinear partial differential equations (PDEs) are hot topical issues, and different algorithms proposed in the literature show efficient numerical approximation in high dimension. In this paper, we introduce a class of PDEs that are invariant to permutations, and called symmetric PDEs. Such problems are widespread, ranging from cosmology to quantum mechanics, and option pricing/hedging in multi-asset market with exchangeable payoff. Our main application comes actually from the particles approximation of mean-field control problems. We design deep learning algorithms based on certain types of neural networks, named PointNet and DeepSet (and their associated derivative networks), for computing simultaneously an approximation of the solution and its gradient to symmetric PDEs. We illustrate the performance and accuracy of the PointNet/DeepSet networks compared to classical feedforward ones, and provide several numerical results of our algorithm for the examples of a mean-field systemic risk, mean-variance problem and a min/max linear quadratic McKean-Vlasov control problem.

Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions

We study the problem of learning-augmented predictive linear quadratic control. Our goal is to design a controller that balances consistency, which measures the competitive ratio when predictions are accurate, and robustness, which bounds the competitive ratio when predictions are inaccurate.

Suboptimal linear quadratic tracking control for multi-agent systems

This paper investigates the distributed linear quadratic tracking control problem for multi-agent leader–follower systems. Considering that only few followers can access the state information of the leader owing to the limited communication range, a novel distributed control law is designed to make followers convergent to the leader by introducing appropriate interconnections among the followers. Besides the tracking consensus of the leader–follower systems, the designed control law also enables the associated cost to be less than a given tolerance for any initial states of the leader and the followers. The implementation of the designed distributed control law can be executed by solving a single Riccati equation, requiring no global information of the communication topology. Two examples are finally provided to demonstrate the effectiveness of the proposed method.

A Koopman operator approach for the vertical stabilization of an off-road vehicle

Vehicles traversing off-road terrains experience modes of dynamics that are not usually encountered on-road. Deformable terrains with significant variations in height and the presence of bumps can lead to significant vertical dynamics in a vehicle. Besides rider discomfort and safety of the vehicle, in the case of unmanned vehicles, these vertical dynamics coupled with pitch motion can produce significant disturbances to sensors such as cameras. Reducing these vertical dynamics is therefore important. However any control design for this problem has to first contend with the modeling complexity of vehicle-tire-terrain interaction. Motivated by recent developments in dynamical systems, we propose the use of the Koopman operator to obtain a linear representation for both the vehicle and terrain interaction dynamics as well as the effect of the control input. As a test case for this framework we address the problem of stabilizing the vertical dynamics of a half-car model moving on a deformable terrain using a propulsive force on the car as the sole control input. The terramechanics are modeled using the Bekker-Wong equations. Using the Koopman operator framework, we obtain a lifted, linear representation of the nonlinear control system, which is then used to formulate the optimal control problem as a constrained linear quadratic model predictive control problem on the lifted system. The framework proposed in this paper can potentially be extended to design a combination of data-driven and physics-based control algorithms for intelligent suspensions, motion control and path-tracking for unmanned ground vehicles on unstructured terrains.

Optimal control in poroelasticity

In this paper we address optimal control problems subject to fluid flows through deformable, porous media. In particular, we focus on linear quadratic elliptic-parabolic control problems, with both distributed and boundary controls, and prove existence and uniqueness of optimal control. Furthermore, we derive the first order necessary optimality conditions. These problems have important biological and biomechanical applications. For example, optimizing the pressure of the flow, and investigating the influence and control of pertinent biological parameters are relevant in the case of the lamina cribrosa – a porous tissue at the base of the optic nerve head inside the eye – which is modeled by poroelasticity – where these factors are believed to be related to the development of ocular neurodegenerative diseases such as glaucoma. Moreover, the study and results will be applicable to other situations such as the poroelastic modeling of cartilages, bones, and engineered tissues.

Channel modeling and LQG control in the presence of random delays and packet drops

We study the modeling and control for discrete-time networked control systems (NCSs) over the random delay and packet drop channel. By analyzing the relation between delays and data packets, the notion of channel states is proposed and conditional probability distributions are employed to model the channel. The channel states are characterized by the finite Markov states, enabling the computation of the corresponding transition probability matrix. As the number of Markov states increases exponentially with respect to the delay bound, the reduced channel states are proposed and studied to ease the complexity issue in the controller design. The linear quadratic Gaussian control problem is then studied for the NCS in which random delays and packet drops exist at both the plant input and output. The controller design problem is converted into state feedback control and state estimation for the augmented system of the plant and channel models. Under the framework of Markovian jump linear systems, the optimal control laws, which are non-causal, are obtained, implemented by transmitting multiple control data in a single packet. An optimal jump linear estimator that has a recursive form is also derived. Finally, comparative simulations are provided to demonstrate the effectiveness of the proposed modeling and design methods.

Linear-quadratic control for a class of stochastic Volterra equations: Solvability and approximation

We provide an exhaustive treatment of linear-quadratic control problems for a class of stochastic Volterra equations of convolution type, whose kernels are Laplace transforms of certain signed matrix measures which are not necessarily finite. These equations are in general neither Markovian nor semimartingales, and include the fractional Brownian motion with Hurst index smaller than 1/2 as a special case. We establish the correspondence of the initial problem with a possibly infinite dimensional Markovian one in a Banach space, which allows us to identify the Markovian controlled state variables. Using a refined martingale verification argument combined with a squares completion technique, we prove that the value function is of linear quadratic form in these state variables with a linear optimal feedback control, depending on nonstandard Banach space valued Riccati equations. Furthermore, we show that the value function of the stochastic Volterra optimization problem can be approximated by that of conventional finite dimensional Markovian linear-quadratic problems, which is of crucial importance for numerical implementation.

Strong error estimates for a space-time discretization of the linear-quadratic control problem with the stochastic heat equation with linear noise

Abstract We propose a time-implicit, finite-element-based space-time discretization of the necessary and sufficient optimality conditions for the stochastic linear-quadratic optimal control problem with the stochastic heat equation driven by linear noise of type $[X(t)+\sigma (t)]\,\,\textrm{d}W(t)$ and prove optimal convergence w.r.t. both space and time discretization parameters. In particular, we employ the stochastic Riccati equation as a proper analytical tool to handle the linear noise, and thus extend the applicability of the earlier work by Prohl & Wang (2021, Strong rates of convergence for a space-time discretization of the backward stochastic heat equation, and of a linear-quadratic control problem for the stochastic heat equation. ESAIM Control Optim. Calc. Var., 27, 54), where the error analysis was restricted to additive noise.

Error analysis of a discretization for stochastic linear quadratic control problems governed by SDEs

Abstract In this work, a time-implicit discretization for stochastic linear quadratic problems subject to stochastic differential equations with control-dependence noises is proposed, and the convergence rate of this discretization is proved. Compared to the existing results, the control variables are stochastic processes and can be contained in systems’ diffusion term. Based on this discretization, a gradient descent algorithm and its convergence rate are presented. Finally, a numerical example is provided to support the theoretical finding.

Robust control in a rough environment

This paper studies robust decision problems in a rough environment that is described by Volterra processes. Various alternative dominated models are introduced to reflect decision makers' concerns regarding model uncertainty. Using a functional Itô calculus approach, we characterize the robust optimal strategy by a path-dependent Hamilton–Jacobi–Bellman–Isaacs equation. Explicit strategies are derived for robust power and exponential utility maximization with the Volterra Heston model and an example from a robust linear-quadratic control problem. Numerical study shows that it becomes harder to distinguish two probability measures in a rougher environment. Robust optimal investment strategies can reduce losses from ignoring volatility roughness and model uncertainty and can improve the stability of portfolio performance.

Minimum energy with infinite horizon: From stationary to non-stationary states

We study a non-standard infinite horizon, infinite dimensional linear–quadratic control problem arising in the physics of non-stationary states (see e.g. Bertini et al. (2004, 2005)): finding the minimum energy to drive a given stationary state x̄=0 (at time t=−∞) into an arbitrary non-stationary state x (at time t=0). This is the opposite to what is commonly studied in the literature on null controllability (where one drives a generic state x into the equilibrium state x̄=0). Consequently, the Algebraic Riccati Equation (ARE) associated with this problem is non-standard since the sign of the linear part is opposite to the usual one and since its solution is intrinsically unbounded. Hence the standard theory of AREs does not apply. The analogous finite horizon problem has been studied in the companion paper (Acquistapace and Gozzi, 2017). Here, similarly to such paper, we prove that the linear selfadjoint operator associated with the value function is a solution of the above mentioned ARE. Moreover, differently to Acquistapace and Gozzi (2017), we prove that such solution is the maximal one. The first main result (Theorem 5.8) is proved by approximating the problem with suitable auxiliary finite horizon problems (which are different from the one studied in Acquistapace and Gozzi (2017)). Finally in the special case where the involved operators commute we characterize all solutions of the ARE (Theorem 6.5) and we apply this to the Landau–Ginzburg model.

Necessary conditions for partially observed optimal control of general McKean–Vlasov stochastic differential equations with jumps

In this paper, we establish necessary conditions of optimality for partially observed optimal control problems of Mckean–Vlasov type. The system is described by a controlled stochastic differential equation governed by Poisson random measure and an independent Brownian motion. The coefficients of the McKean–Vlasov system depend on the state of the solution process as well as of its probability law and the control variable. The proof of our result is based on Girsanov's theorem, variational equations and derivatives with respect to probability measure under convexity assumption. At the end of this paper, we apply our stochastic maximum principle to study partially observed linear quadratic control problem of McKean–Vlasov type with jumps and derive the explicit expression of the optimal control.

Asymmetric information control for stochastic systems with different intermittent observations

This paper will investigate the linear quadratic (LQ) control problem for a stochastic system with different intermittent observations. Different from traditional LQ control problems, two controllers with different information structures are considered to minimize a quadratic performance index. Specifically, the information sets available to two controllers are different and mutually exclusive (i.e., either of both is not the subset of the other one). The problem under consideration in this paper is an asymmetric information control problem. It is highlighted that previous literature on asymmetric information control mainly focused on the case when the information set is a subset of the other one. However, the case of the information sets being mutually exclusive remains less investigated to the best of our knowledge, which brings essential difficulties in finding the optimal control strategies: the structures and forms of two controllers are not clear; seeking for the solvability conditions of the optimal control problem is challenging. We summarize the contributions of this paper as follows: Firstly, by adopting the convex variational method, the necessary and sufficient solvability conditions are derived under some basic assumptions, which are relied on the forward and backward difference equations (FBSDEs); Secondly, by decoupling the FBSDEs, the control strategies are derived; Finally, the results are extended to investigate the general multiple controllers case.

A linear implicit Euler method for the finite element discretization of a controlled stochastic heat equation

AbstractWe consider a numerical approximation of a linear quadratic control problem constrained by the stochastic heat equation with nonhomogeneous Neumann boundary conditions. This involves a combination of distributed and boundary control, as well as both distributed and boundary noise. We apply the finite element method for the spatial discretization and the linear implicit Euler method for the temporal discretization. Due to the low regularity induced by the boundary noise, convergence orders above $1/2$ in space and $1/4$ in time cannot be expected. We prove such optimal convergence orders for our full discretization when the distributed noise and the initial condition are sufficiently smooth. Under less smooth conditions the convergence order is further decreased. Our results only assume that the related (deterministic) differential Riccati equation can be approximated with a certain convergence order, which is easy to achieve in practice. We confirm these theoretical results through a numerical experiment in a two-dimensional domain.

Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems

<p style='text-indent:20px;'>In this paper, we consider linear-quadratic (LQ) leader-follower Stackelberg differential games for mean-field type stochastic systems with jump diffusions, where the system includes mean-field variables, i.e., the expected value of state and control variables. We first solve the LQ mean-field type control problem of the follower using the stochastic maximum principle and obtain the state-feedback representation of the open-loop optimal solution in terms of the coupled integro-Riccati differential equations (CIRDEs) via the Four-Step Scheme. Next, we solve the problem of the leader, which is the LQ control problem subject to the mean-field type forward-backward stochastic system with jump diffusions, where the constraint characterizes the rational behavior of the follower. Using the variational approach, we obtain the (mean-field type) stochastic maximum principle. However, to obtain the state-feedback representation of the open-loop optimal solution of the leader, there is a technical challenge due to the jump process. We consider two different cases, in which the state-feedback type control in terms of the CIRDEs can be characterized by generalizing the Four-Step Scheme. We finally show that the state-feedback type controls of the open-loop optimal solutions for the leader and the follower constitute the Stackelberg equilibrium.</p>