Linear Quadratic Problem Research Articles

Strongly continuous quasi semigroups in optimal control problems for non-autonomous systems

This paper addresses linear quadratic control optimal problems for non-autonomous linear control systems using strongly continuous quasi semigroups. Riccati equations are implemented to investigate the control optimal problems of cost functional with finite and infinite horizons. The unique optimal pair for the cost functional is determined by the mild solution of the associated closed-loop problem and the feedback control of the solution of the corresponding Riccati equation. In addition for the infinite horizon, the stabilizability of the system is a sufficiency for the solvability to the Riccati equation. An application in a parabolic system is proposed.

Time-Inconsistent Stochastic LQ Problem with Regime Switching

This paper investigates a time-inconsistent stochastic linear-quadratic problem with regime switching that is characterized via a finite-state Markov chain. Open-loop equilibrium control is studied in this paper whose existence is characterized via Markov-chain-modulated forward-backward stochastic difference equations and generalized Riccati-like equations with jumps.

Output Feedback Q-Learning for Linear-Quadratic Discrete-Time Finite-Horizon Control Problems.

An algorithm is proposed to determine output feedback policies that solve finite-horizon linear-quadratic (LQ) optimal control problems without requiring knowledge of the system dynamical matrices. To reach this goal, the Q -factors arising from finite-horizon LQ problems are first characterized in the state feedback case. It is then shown how they can be parameterized as functions of the input-output vectors. A procedure is then proposed for estimating these functions from input/output data and using these estimates for computing the optimal control via the measured inputs and outputs.

Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach

Standard stochastic control methods assume that the probability distribution of uncertain variables is available. Unfortunately, in practice, obtaining accurate distribution information is a challenging task. To resolve this issue, in this article we investigate the problem of designing a control policy that is robust against errors in the empirical distribution obtained from data. This problem can be formulated as a two-player zero-sum dynamic game problem, where the action space of the adversarial player is a Wasserstein ball centered at the empirical distribution. A dynamic programming solution is provided exploiting the reformulation techniques for Wasserstein distributionally robust optimization. We show that the contraction property of associated Bellman operators extends a single-stage out-of-sample performance guarantee, obtained using a measure concentration inequality, to the corresponding multistage guarantee without any degradation in the confidence level. Furthermore, we characterize an explicit form of the optimal control policy and the worst-case distribution policy for linear-quadratic problems with Wasserstein penalty.

Convex Model for Controlled Islanding in Transmission Expansion Planning to Improve Frequency Stability

Intentional controlled islanding (ICI) is the last resort to split an endangered power system into smaller islands to prevent blackout. New lines that are planned by transmission expansion planning (TEP) can affect the stability of islands during ICI. In this paper, an ICI-TEP method is proposed to improve the stability of islands by more efficient planning of transmission assets. Moreover, by developing a criterion for the frequency of center of inertia (COI) in each island, the frequency deviations of generators from the COI frequency are minimized to result in more stable islands. The proposed ICI-TEP, incorporating AC network representation, is modeled as mixed-integer linear programming and quadratic convex problems ensuring tractability. A Benders decomposition strategy is also proposed to solve the problem. Results of testing the proposed ICI-TEP method on IEEE 39-bus and 300-bus test systems confirm its effectiveness, compared to conventional TEP, in terms of coping with sever disturbances by creating more stable islands with a lower load shedding.

A symplectic pseudospectral method for constrained time-delayed optimal control problems and its application to biological control problems

ABSTRACT Time-delays are often involved in modelling biological processes, which arise more difficulties for the control task. In fact, once the control goal is given, a constrained optimal control problem can be formulated to describe the biological control mission. However, it’s almost impossible to obtain analytical solutions for such complicated problems, especially for those with high nonlinearity and complex constraints. In this paper, a symplectic pseudospectral method for solving nonlinear optimal control problems of constrained stated-delayed system is developed. The original nonlinear problem is transformed into a series of linear-quadratic optimal control problems by the successive convexification technique at the first implementation. By applying the local Legendre-Gauss-Lobatto pseudospectral method and the parametric variational principle, each of the transformed linear-quadratic problems is converted into a standard linear complementarity problem which can be solved easily by the Lemke’s method. The cost functional, system dynamics and constraints can be generally nonlinear functions of state, control and time variables. and three kinds of inequality constraints, i.e. pure-state, pure-control and mixed state-control, can be treated under a uniform framework. The developed symplectic pseudospectral method is finally validated by an optimal harvesting problem and an optimal vaccination problem.

On the relation between turnpike properties and dissipativity for continuous time linear quadratic optimal control problems

<p style='text-indent:20px;'>The paper is devoted to analyze the connection between turnpike phenomena and strict dissipativity properties for continuous-time finite dimensional linear quadratic optimal control problems. We characterize strict dissipativity properties of the dynamics in terms of the system matrices related to the linear quadratic problem. These characterizations then lead to new necessary conditions for the turnpike properties under consideration, and thus eventually to necessary and sufficient conditions in terms of spectral criteria and matrix inequalities. One of the key novelty of these results is the possibility to encompass the presence of state and input constraints.

Energy-Driven-Damper (EDD): Comfort-Oriented Semiactive Suspensions Optimized From an Energy Perspective

This brief reports that comfort performance of semiactive suspensions can be optimized by building a controller driven by the kinetic energy of the sprung mass. By solving a linear quadratic regulator problem pursuing minimal energy input and maximal energy output, the energy-driven-damper (EDD) controller is proposed. An insightful interpretation from an energy point of view is given, which helps to reveal the design mechanism of an optimal semiactive suspension controller. Using the most-used Skyhook (SH), acceleration-driven damper (ADD), and mixed SHADD as the benchmark, the effectiveness and superiority of EDD are elucidated based on CarSim-MATLAB cosimulations, where a nonlinear magnetorheological (MR) damper model is adopted.

Dynamic Equilibrium with Randomly Arriving Players

There are real strategic situations where nobody knows ex ante how many players there will be in the game at each step. Assuming that entry and exit could be modeled by random processes whose probability laws are common knowledge, we use dynamic programming and piecewise deterministic Markov decision processes to investigate such games. We study these games in discrete and continuous time for both finite and infinite horizon. While existence of dynamic equilibrium in discrete time is proved, our main aim is to develop algorithms. In the general nonlinear case, the equations provided are rather intricate. We develop more explicit algorithms for both discrete and continuous time linear quadratic problems.

Optimal preview repetitive control with application to permanent magnet synchronous motor drive system

In this paper, the problem of optimal preview repetitive control (OPRC) for continuous-time linear systems is proposed. First, an augmented dynamic system including the derivative of the state vector and the output of the basic repetitive controller is constructed by using lift technique. Then, a new performance index is denoted, which transforms the tracking problem into a linear quadratic regulation problem. Based on the optimal control theory, the explicit optimal preview repetitive controller is obtained, which combination of the state feedback, the integral action related to the tracking error, the basic repetitive controller and the preview compensation. Finally, the developed design techniques are applied to permanent magnet synchronous motor (PMSM) speed control problem. The simulation results demonstrate the validity of the proposed method.

Continuous-time inverse quadratic optimal control problem

In this paper, the problem of finite horizon inverse optimal control (IOC) is investigated, where the quadratic cost function of a dynamic process is required to be recovered based on the observation of optimal control sequences. We propose the first complete result of the necessary and sufficient condition for the existence of corresponding standard linear quadratic (LQ) cost functions. Under feasible cases, the analytic expression of the whole solution space is derived and the equivalence of weighting matrices in LQ problems is discussed. For infeasible problems, an infinite dimensional convex problem is formulated to obtain a best-fit approximate solution with minimal control residual. And the optimality condition is solved under a static quadratic programming framework to facilitate the computation. Finally, numerical simulations are used to demonstrate the effectiveness and feasibility of the proposed methods.

Quantification of operating reserves with high penetration of wind power considering extreme values

The high integration of wind energy in power systems requires operating reserves to ensure the reliability and security in the operation. The intermittency and volatility in wind power sets a challenge for day-ahead dispatching in order to schedule generation resources. Therefore, the quantification of operating reserves is addressed in this paper using extreme values through Monte-Carlo simulations. The uncertainty in wind power forecasting is captured by a generalized extreme value distribution to generate scenarios. The day-ahead dispatching model is formulated as a mixed-integer linear quadratic problem including ramping constraints. This approach is tested in the IEEE-118 bus test system including integration of wind power in the system. The results represent the range of values for operating reserves in day-ahead dispatching.

Optimization of Constrained Stochastic Linear-Quadratic Control on an Infinite Horizon: A Direct-Comparison Based Approach

In this paper we study the optimization of the discrete-time stochastic linear-quadratic (LQ) control problem with conic control constraints on an infinite horizon, considering multiplicative noises. Stochastic control systems can be formulated as Markov Decision Problems (MDPs) with continuous state spaces and therefore we can apply the direct-comparison based optimization approach to solve the problem. We first derive the performance difference formula for the LQ problem by utilizing the state separation property of the system structure. Based on this, we successfully derive the optimality conditions and the stationary optimal feedback control. By introducing the optimization, we establish a general framework for infinite horizon stochastic control problems. The direct-comparison based approach is applicable to both linear and nonlinear systems. Our work provides a new perspective in LQ control problems; based on this approach, learning based algorithms can be developed without identifying all of the system parameters.

Optimal universal controllers for roll stabilization

Roll stabilization is an important problem of ship motion control. This problem becomes especially difficult if the same set of actuators (e.g. a single rudder) has to be used for roll stabilization and heading control of the vessel, so that the roll stabilizing system interferes with the ship autopilot. Finding the “trade-off” between the concurrent goals of accurate vessel steering and roll stabilization usually reduces to an optimization problem, which has to be solved in presence of an unknown wave disturbance. Standard approaches to this problem (loop-shaping, LQG, H∞-control etc.) require to know the spectral density of the disturbance, considered to be a “colored noise”. In this paper, we propose a novel approach to optimal roll stabilization, approximating the disturbance by a polyharmonic signal with known frequencies yet uncertain amplitudes and phase shifts. Linear quadratic optimization problems in presence of polyharmonic disturbances can be solved by means of the theory of universal controllers developed by V.A. Yakubovich. An optimal universal controller delivers the optimal solution for any uncertain amplitudes and phases. Using Marine Systems Simulator (MSS) Toolbox that provides a realistic vessel's model, we compare our design method with classical approaches to optimal roll stabilization. Among three controllers providing the same quality of yaw steering, OUC stabilizes the roll motion most efficiently.

Infinite time linear quadratic stackelberg game problem for unknown stochastic discrete‐time systems via adaptive dynamic programming approach

AbstractIn this paper, we propose an adaptive dynamic programming (ADP) approach to solve the infinite horizon linear quadratic (LQ) Stackelberg game problem for unknown stochastic discrete‐time systems with multiple decision makers. Firstly, the stochastic LQ Stackelberg game problem is converted into the deterministic problem by system transformation. Next, a value iteration ADP approach is put forword and the convergence is given. Thirdly, in order to implement the iterative method, back propagation neural network (BPNN) is chosen to design model network, critic network and action network to approximate the unknown systems, objective functions and Stackelberg strategies. Finally, simulation results show that the algorithm is effective.

On closed-loop equilibrium strategies for mean-field stochastic linear quadratic problems

This article is concerned with linear quadratic optimal control problems of mean-field stochastic differential equations (MF-SDE) with deterministic coefficients. To treat the time inconsistency of the optimal control problems, linear closed-loop equilibrium strategies are introduced and characterized by variational approach. Our developed methodology drops the delicate convergence procedures in Yong [Trans. Amer. Math. Soc.369(2017) 5467–5523]. When the MF-SDE reduces to SDE, our Riccati system coincides with the analogue in Yong [Trans. Amer. Math. Soc.369(2017) 5467–5523]. However, these two systems are in general different from each other due to the conditional mean-field terms in the MF-SDE. Eventually, the comparisons with pre-committed optimal strategies, open-loop equilibrium strategies are given in details.

Data-driven parameterizations of suboptimal LQR and H2 controllers

In this paper we design suboptimal control laws for an unknown linear system on the basis of measured data. We focus on the suboptimal linear quadratic regulator problem and the suboptimal H2 control problem. For both problems, we establish conditions under which a given data set contains sufficient information for controller design. We follow up by providing a data-driven parameterization of all suboptimal controllers. We will illustrate our results by numerical simulations, which will reveal an interesting trade-off between the number of collected data samples and the achieved controller performance.

Data-driven Linear Quadratic Regulation via Semidefinite Programming

Abstract This paper studies the finite-horizon linear quadratic regulation problem where the dynamics of the system are assumed to be unknown and the state is accessible. Information on the system is given by a finite set of input-state data, where the input injected in the system is persistently exciting of a sufficiently high order. Using data, the optimal control law is then obtained as the solution of a suitable semidefinite program. The effectiveness of the approach is illustrated via numerical examples.

Chance-constrained LQG production planning problem under partially observed forward-backward inventory systems.

Abstract The target system of our research is a reverse logistics system with imperfect information of inventory variables. This system is affected by two independents and uncorrelated random variables that represent demand and return fluctuations. A Discrete-time, chance-constrained, Linear Quadratic Gaussian Problem under imperfect information of inventory systems (DCLQG) is formulated in order to develop an aggregate manufacturing and remanufacturing plan. Technically, an optimal closed-loop solution for this stochastic problem is possible, but it is not easy to get it, particularly for large size problems. Thus, an open-loop updating approach that provides a quasi-optimal solution is investigated here. This approach considers an equivalent deterministic problem to the DCLQG problem. It is based on the conditional mean value and on variances of inventory variables, which are estimated from a Kalman filter procedure. Such an approach allows managers to build an aggregated production plan, periodically revised, that helps them to make decisions. An open-loop updating approach is compared to a no-updating approach, which depends only on the initial condition of states of the system. An example shows the importance of information gathering to provide sub-optimal solutions for stochastic problems with imperfect information of states. It is also shown that sub-optimal production policies can improve the company’s profitability.

Performance Assessment of FO-PID Temperature Control System Using a Fractional Order LQG Benchmark

In this paper, a fractional order LQG benchmark is proposed for the control performance assessment of fractional order control systems. Similar to the conventional LQG benchmark, the fractional order LQG performance benchmark curve is determined by the numerical calculation method, which avoids the calculation of the complex interaction matrix. The fractional order process model is discretized via fractional order calculus. Meanwhile, the fractional order integral is introduced into the conventional LQG cost function. Then solving the linear quadratic Gaussian problem under the fractional order model and fractional control, the optimal input and output variances are determined for different weighting factors and the performance curves can be achieved. The comparison between fractional order LQG and the conventional LQG shows the improvement of the proposed benchmark under the same condition. The proposed benchmark can provide a more direct and superior reference standard to evaluate the performance of fractional order control system. Finally, a case study of fractional order PID(FO-PID) controller in industrial heating furnace temperature control experiment with model matching and model mismatch conditions is used to verify the effectiveness of the proposed benchmark.