Optimal Control Of Discrete-time Systems Research Articles

Deterministic optimal control for discrete-time systems with multiplicative noises and random coefficients

This paper studies a linear-quadratic optimal control problem for discrete-time systems with multiplicative noise and random coefficients. Motivated by fiscal policy planning problems, where deterministic policies are required, this paper aims to find a deterministic optimal controller. The challenge is to seek a solution to forward and backward stochastic difference equations with structural inconsistency caused by random and deterministic terms. We derive necessary and sufficient conditions to solve this optimal control problem by employing a decoupling method. We also present an explicit expression of the optimal controller based on coupled stochastic Riccati-type difference equations.

Model-free optimal control of discrete-time systems with additive and multiplicative noises

This paper investigates the optimal control problem for a class of discrete-time stochastic systems subject to additive and multiplicative noises. An algebraic Riccati equation is established which gives the form of the solution to the problem. To obtain the optimal control gain iteratively, an offline policy iteration is presented with convergence proof. A model-free reinforcement learning algorithm is proposed to learn the optimal admissible control policy using the system states and inputs without resorting to the system matrices. It is proven that the estimation error of the kernel matrix is bounded and the iterative control gain is admissible. Compared with the existing work, this paper considers the model-free controller learning problem for stochastic systems suffering from both additive and multiplicative noises using reinforcement learning. The proposed algorithm is illustrated through a numerical example, which shows that our algorithm outperforms other policy iteration algorithms.

Mixed optimal control for discrete-time stochastic systems with random coefficients

This paper is concerned with the problem of mixed optimal control for discrete-time systems with random coefficients where “mixed” means that there are two controllers: the deterministic controller and the random controller. Different from previous works, random coefficients and controllers with different information structures arise simultaneously in the optimal control problem, which lead to large difficulty. Using decoupling technique to solve the forward and backward stochastic difference equations, the necessary and sufficient conditions for the solvability of optimal control problem are given. An explicit expression of the optimal control strategies is derived based on a stochastic Riccati-type equation, which is new to the best of knowledge. Finally, a pension management problem is solved with the obtained theoretical results.

Robust induced [formula omitted] optimal control of discrete-time systems having magnitude and rate-bounded actuators

The design of robust state- and output-feedback control for uncertain discrete-time systems with physical magnitude and rate constraints on their actuator dynamics was addressed. Unlike the traditional methods such as anti-windup (AW) methods, nested ellipsoids, model predictive controllers (MPCs) and integral quadratic constraints(IQCs) formulated by sector bounded inequalities, this paper uses a transformation of the system dynamics to a form which considers control signal and its rate as controlled outputs and using discrete-time ℓ∞ induced (peak-to-peak) norm from disturbance inputs to these outputs. To cope with the magnitude and rate bound non-linearities together, the induced ℓ∞ norm from disturbance input to the outputs involving control signal and its rate is utilised. On the other hand, discrete-time(DT) induced ℓ2 norm from disturbance input to the main controlled output is used to mitigate the effects of disturbances. We can tackle this ambitious non-linear control problem in the domain of linear convex multi-objective optimal control problem, which can be solved by effective semi-definite optimisation methods by using the proposed transformation and handling the control constraints in terms of worst case peak-to-peak gain of the system. Extended Linear Matrix Inequalities (LMIs) and full block S-procedure based design conditions developed over Linear Fractional Representation(LFR) framework allow the user to obtain robust state- and output-feedback control solutions with reduced conservatism. For the first time, this paper introduces an extended LMI based robust output-feedback control design for magnitude and rate bounded (MRB) systems, using full block S-procedure. We demonstrate the performance of the proposed controller through several simulations over benchmark examples covering systems having multi-variable structures and uncertainties. Our study also involves comparison results with a recently introduced technique based on multi-stage AW technique. The simulation results show that the proposed method of this paper is much effective and less conservative compared to the recent AW method provided in the literature.

Optimal Control With Maximum Cost Performance Measure

Optimal control of discrete-time systems with a less common performance measure is investigated, in which the cost function to be minimized is the maximum, instead of the sum, of a cost per stage over the control time. Three control scenarios are studied under a finite-horizon, a discounted infinite-horizon, and an undiscounted infinite-horizon performance measure. For each case, the Bellman equation is derived by direct use of dynamic programming, and the necessary and sufficient conditions for an optimal control are established around this equation. A motivating example on optimal control of dc-dc buck power converters is presented.

Stabilization and optimal control of discrete-time systems with multiplicative noise and multiple input delays

This paper is concerned with the stabilization and optimal control of discrete-time systems with multiplicative noise and multiple input delays. First, it is shown that the systems are stabilizable if and only if some algebraic Riccati-type equations have a solution which satisfies a given condition. To the best of our knowledge, this is the first time to propose a necessary and sufficient stabilizing condition for general multiple-input delay systems with multiplicative noise. The method constructs a Lyapunov–Krasovskii function based on the optimal cost of a finite-horizon LQR problem. Next, under the assumption that the system is stabilizable, an infinite-horizon LQR problem is investigated. The optimal control and the optimal cost are given by the above algebraic Riccati-type equations. Finally, based on the above stabilizing condition, an exact delay range for the stabilization of the system is derived in a special case.

Invariant Adaptive Dynamic Programming for Discrete-Time Optimal Control

For systems that can only be locally stabilized, control laws and their effective regions are both important. In this paper, invariant policy iteration is proposed to solve the optimal control of discrete-time systems. At each iteration, a given policy is evaluated in its invariantly admissible region, and a new policy and a new region are updated for the next iteration. Theoretical analysis shows the method is regionally convergent to the optimal value and the optimal policy. Combined with sum-of-squares polynomials, the method is able to achieve the near-optimal control of a class of discrete-time systems. An invariant adaptive dynamic programming algorithm is developed to extend the method to scenarios where system dynamics is not available. Online data are utilized to learn the near-optimal policy and the invariantly admissible region. Simulated experiments verify the effectiveness of our method.

Observer-based adaptive neural optimal control for discrete-time systems in nonstrict-feedback form

This paper considers the problem of observer-based adaptive near-optimal control for a class of nonstrict-feedback discrete-time nonlinear systems with non-symmetric dead zone. In order to compensate the effect of dead-zone on the control performance, an adaptive auxiliary signal is constructed to estimate the unknown dead-zone parameters. For the unknown nonlinear functions, neural networks (NNs) are introduced to identify them and to estimate the unknown parameters. To resolve the difficulty resulting from the unavailable state variables, an NN-based observer is designed. Moreover, according to the framework of adaptive control, a novel reinforcement learning algorithm is developed to guarantee that the near-optimal control performance is achieved. Finally, some simulation results are provided to illustrate the effectiveness of the proposed control algorithm.

Discrete-time [formula omitted] optimal control under intermittent and lossy communications

The paper studies the H2 optimal control for discrete-time systems under the constraint that the information exchange between the sensor- and actuator-side parts of the controller is intermittent, a priori unknown, and independent of the process. A closed-form analytic solution to the problem is derived. The solution is based on two standard algebraic Riccati equations and is readily implementable. An implementation maintaining optimality of the controller under packet losses, which can be modeled by the intermittent communication framework, is also proposed. On the technical side, the paper reveals an issue related to the need for extending the time-varying error system to the whole discrete time axis Z in order to solve the corresponding H2 model-matching problem and proposes a procedure to resolve it.

Problems of Adaptive Optimal Control of Discrete-Time Systems under Bounded Disturbance and Linear Performance Indexes

We present some problems of adaptive optimal robust control of linear discretetime systems under uncertainty and bounded external disturbance in which the optimal or guaranteed value of the performance index is a linear or linear-fractional function of unknown parameters of the system and set-membership estimation based on the method of recurrent objective inequalities reduces to recurrent updating of polyhedral estimates of unknown parameters. In such problems, computing current optimal estimates becomes a recurrent linear programming problem which is computationally tractable on modern computers for systems with a small number of estimated parameters.

Polynomial J-spectral factorization method for optimal control of discrete-time systems

We present a simple algorithm for linear-quadratic control of discrete-time systems, based on polynomial J-spectral factorization.

Convergence Proof of Approximate Policy Iteration for Undiscounted Optimal Control of Discrete-Time Systems

Approximate policy iteration (API) is studied to solve undiscounted optimal control problems in this paper. A discrete-time system with the continuous-state space and the finite-action set is considered. As approximation technique is used for the continuous-state space, approximation errors exist in the calculation and disturb the convergence of the original policy iteration. In our research, we analyze and prove the convergence of API for undiscounted optimal control. We use an iterative method to implement approximate policy evaluation and demonstrate that the error between approximate and exact value functions is bounded. Then, with the finite-action set, the greedy policy in policy improvement is generated directly. Our main theorem proves that if a sufficiently accurate approximator is used, API converges to the optimal policy. For implementation, we introduce a fuzzy approximator and verify the performance on the puddle world problem.

Decentralized control of discrete-Time system with delay in mean field LQR problem

This paper studies the decentralized optimal control of discrete-time system with inputdelay, where a large number of agents with the identical decoupling dynamical equations and the coupling cost function through the mean field are considered. The decentralized and centralized optimal controllers are proposed by the optimal tracking control of LQR problem with delay. They are proved that the optimal controllers and the optimal cost function of the centralized and decentralized solutions are equivalent for the optimal control problem. An illustrative example is given to show the efficiency of the decentralized optimal controllers.

PALINDROMIC EIGENVALUE PROBLEMS: A BRIEF SURVEY

The T-palindromic quadratic eigenvalue problem $(\lambda^2 B + \lambda C + A)x = 0$, with $A,B,C \in \mathbb{C}^{n \times n}$, $C^T = C$ and $B^T = A$, governs the vibration behaviour of trains. Other palindromic eigenvalue problems, quadratic or higher order, arise from applications in surface acoustic wave filters, optimal control of discrete-time systems and crack modelling. Numerical solution of palindromic eigenvalue problems is challenging, with unacceptably low accuracy from the basic linearization approach. In this survey paper, we shall talk about the history of palindromic eigenvalue problems, in terms of their history, applications, numerical solution and generalization. We shall also speculate on some future directions of research.

A coprime factorisation and its application in ℋ2 control

Given a plant transfer matrix as a fraction of two polynomial matrices, we present a doubly coprime factorisation identity over the proper stable rational matrices of the plant transfer matrix, and apply it to compute a controller transfer matrix in ℋ2 optimal control of discrete-time systems. If there are no uncontrollable modes of the plant transfer matrix, the controller is given by a fraction of two polynomial matrices, where the order of the denominator is less than or equal to the McMillan degree of the plant.

On Stabilization of Optimal Control of Discrete-Time Systems

The paper deals with the problem of the stabilizing optimal control for the multivariable discrete-time stochastic systems in the case of the quadratic optimality criterion. The stability of the closed control loop is ensured by the proper choice of the terminal state penalization matrix in the optimality criterion, i.e. the initial condition for the considered computational algorithm based on the dynamic programming. In addition, the choice of this matrix presented in the paper is also suitable in the cases when the nonminimal realization of the system is of interest. Then the reduction of the calculated matrix dimension is possible when using the square root algorithm of the computation. All the results obtained for the state-space system model are applied to the analogical problem for the system described by the single-input, single-output (SISO) regression model.

Optimal control of discrete-time systems with time-lag controls

Various optimal control problems for discrete-time systems with time-lag controls are discussed. Some of the basic features of this type of system are noted. A simple example is given for illustrative purpose.

Singular optimal control of discrete-time systems

Singular optimal control of discrete-time systems

Discretional Convexity and the Maximum Principle for Discrete Systems

Directional convexity is a property of sets closely related to, but weaker than, convexity. It is the existence of supporting hyperplanes at all boundary points of a convex set that makes convexity important in optimal control theory. However, there need be supporting hyperplanes on only one side of the sets for much of the development. Directionally convex sets have this property. The concept of directional convexity, a property of sets, is a generalization, in the following sense, of the concept of convexity, a property of functions. The graph of every convex function is a directionally convex set. However, not every directionally convex set is a graph of a function. Since directional convexity is more general than convexity (for both sets and functions), it may unify a method of investigation which uses convex (or concave) functions with another which uses convex sets. Properties of directional convexity and of matrices that preserve directional convexity are given. Directional convexity was introduced recently to extend the applicability of results on the optimal control of discrete-time systems. It is shown here that the results may be further generalized.