Suboptimal State Feedback Research Articles

LQR controller design for affine LPV systems using reinforcement learning

In this paper, a data-driven sub-optimal state feedback is designed for a continuous time linear parameter varying (LPV) system using reinforcement learning. Time-varying parameters lie in a poly top and the system matrix has an affine representation for the parameters. Two novels, on-policy and off-policy algorithms, are proposed using available data from vertex systems of polytop to minimise a performance index and admit a common Lyapunov function (CLF). A convex optimisation problem is derived for each iteration based on Lyapunov inequality. Algorithms yield stabilising feedback gain in each iteration and convergence to a common lyapunov function. We demonstrate the efficacy of the proposed method by simulation of two case studies.

A Robust Vector Current Controller with Negative-Sequence Current Capability for Grid-Connected Inverters

This paper presents a vector current controller (in the synchronous reference, or the dq, frame) with negative-sequence current injection capability for three-phase grid-connected converters. This capability is desired for the operation of the converter during unbalanced conditions and also for a certain type of islanding detection. The proposed controller first determines the double-frequency current references and then uses a sixth-order two-input two-output proportional-integral-resonance (PIR) structure, which is optimally designed. Compared with the existing similar approaches, the proposed controller has a simpler structure and more robust performance, e.g., against system parameter uncertainties and weak grid conditions. The proposed controller is developed for converters with both the L-type and LCL-type filters. For the LCL-type converter, a suboptimal partial state feedback control is also proposed to achieve robust stability and active damping of resonance poles without requiring additional sensors. Detailed experimental results are presented to illustrate the properties and performances of the proposed controller.

Parallel control for optimal tracking via adaptive dynamic programming

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems. Unlike existing optimal state feedback control, the control input of the optimal parallel control is introduced into the feedback system. However, due to the introduction of control input into the feedback system, the optimal state feedback control methods can not be applied directly. To address this problem, an augmented system and an augmented performance index function are proposed firstly. Thus, the general nonlinear system is transformed into an affine nonlinear system. The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically. It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function. Moreover, an adaptive dynamic programming ( ADP ) technique is utilized to implement the optimal parallel tracking control using a critic neural network ( NN ) to approximate the value function online. The stability analysis of the closed-loop system is performed using the Lyapunov theory, and the tracking error and NN weights errors are uniformly ultimately bounded ( UUB ) . Also, the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals. Finally, the effectiveness of the developed optimal parallel control method is verified in two cases.

Decentralized Suboptimal State Feedback Integral Tracking Control Design for Coupled Linear Time-Varying Systems

In this paper, a suboptimal state feedback integral decentralized tracking control synthesis for interconnected linear time-variant systems is proposed by using orthogonal polynomials. Particularly, the use of operational matrices allows, by expanding the subsystem input states and outputs over a shifted Legendre polynomial basis, the conversion of time-varying parameter differential state equations to a set of time-independent algebraic ones. Hence, optimal open-loop state and control input coefficients are forwardly determined. These data are used to formulate a least-square problem, allowing the synthesis of decentralized state feedback integral control gains. Closed-loop asymptotic stability LMI conditions are given. The proposed approach effectiveness is proved by solving a nonconstant reference tracking problem for coupled inverted pendulums.

Data-Driven Model-Free Tracking Reinforcement Learning Control with VRFT-based Adaptive Actor-Critic

This paper proposes a neural network (NN)-based control scheme in an Adaptive Actor-Critic (AAC) learning framework designed for output reference model tracking, as a representative deep-learning application. The control learning scheme is model-free with respect to the process model. AAC designs usually require an initial controller to start the learning process; however, systematic guidelines for choosing the initial controller are not offered in the literature, especially in a model-free manner. Virtual Reference Feedback Tuning (VRFT) is proposed for obtaining an initially stabilizing NN nonlinear state-feedback controller, designed from input-state-output data collected from the process in open-loop setting. The solution offers systematic design guidelines for initial controller design. The resulting suboptimal state-feedback controller is next improved under the AAC learning framework by online adaptation of a critic NN and a controller NN. The mixed VRFT-AAC approach is validated on a multi-input multi-output nonlinear constrained coupled vertical two-tank system. Discussions on the control system behavior are offered together with comparisons with similar approaches.

Switching design for suboptimal guaranteed cost control of 2-D nonlinear switched systems in the Roesser model

This paper deals with the problem of switching design for guaranteed cost control of discrete-time two-dimensional (2-D) nonlinear switched systems described by the Roesser model. The switching signal, which determines the active mode of the system, is subject to a state-dependent process whose values belong to a finite index set. By using 2-D common Lyapunov function approach, a sufficient condition expressed in terms of tractable matrix inequalities is first established to design a min-projection switching rule that makes the 2-D switched system asymptotically stable. The obtained result on stability analysis is then utilized to synthesize a suboptimal state feedback controller that minimizes the upper bound of a given infinite-horizon cost function. Finally, two numerical examples are given to illustrate the effectiveness of the proposed design method.

Distributed Control of the Stochastic Burgers Equation with Random Input Data

AbstractWe discuss a control problem involving a stochastic Burgers equation with a random diffusion coefficient. Numerical schemes are developed, involving the finite element method for the spatial discretisation and the sparse grid stochastic collocation method in the random parameter space. We also use these schemes to compute closed-loop suboptimal state feedback control. Several numerical experiments are performed, to demonstrate the efficiency and plausibility of our approximation methods for the stochastic Burgers equation and the related control problem.

LQ Control of Lotka-Volterra Systems Based on their Locally Linearized Dynamics ,

This work applies the LQ control framework to the class of quasi-polynomial and Lotka-Volterra systems through the linearized version of their nonlinear system model. The primary aim is to globally stabilize the original system with a suboptimal LQ state feedback by means of a well-known entropy-like Lyapunov function that is related to the diagonal stability of linear systems. This aim can only be reached in the case when the quasi-monomial composition matrix is invertible. In the rank-deficient case only the local stabilization of the system is possible with an LQ controller that is designed using the locally linearized model of the closed-loop system model.

Design of hybrid regrouping PSO–GA based sub-optimal networked control system with random packet losses

In this paper, a new approach has been presented to design sub-optimal state feedback regulators over Networked Control Systems (NCS) with random packet losses. The optimal regulator gains, producing guaranteed stability are designed with the nominal discrete time model of a plant using Lyapunov technique which produces a few set of Bilinear Matrix Inequalities (BMIs). In order to reduce the computational complexity of the BMIs, a Genetic Algorithm (GA) based approach coupled with the standard interior point methods for LMIs has been adopted. A Regrouping Particle Swarm Optimization (RegPSO) based method is then employed to optimally choose the weighting matrices for the state feedback regulator design that gets passed through the GA based stability checking criteria i.e. the BMIs. This hybrid optimization methodology put forward in this paper not only reduces the computational difficulty of the feasibility checking condition for optimum stabilizing gain selection but also minimizes other time domain performance criteria like expected value of the set-point tracking error with optimum weight selection based LQR design for the nominal system.

An Enhanced Genetic Programming Algorithm for Optimal Controller Design

This paper proposes a Genetic Programming based algorithm that can be used to design optimal controllers. The proposed algorithm will be named a Multiple Basis Function Genetic Programming (MBFGP). Herein, the main ideas concerning the initial population, the tree structure, genetic operations, and other proposed non-genetic operations are discussed in details. An optimization algorithm called numeric constant mutation is embedded to strengthen the search for the optimal solutions. The results of solving the optimal control for linear as well as nonlinear systems show the feasibility and effectiveness of the proposed MBFGP as compared to the optimal solutions which are based on numerical methods. Furthermore, this algorithm enriches the set of suboptimal state feedback controllers to include controllers that have product time-state terms.

Dynamic state feedback and its application to linear optimal control

We investigate the linear optimal control problems in the context of dynamic state feedback configuration. The dynamic state feedback is a dual structure of the dynamic observer. In conjunction with the well known arguments on linear matrix differential and Lyapunov equations, we elicit the fact that the quadratic performance index is always computable with this configuration. Based on this property, we suggest a nonlinear optimization programming method to get suboptimal or near optimal time-invariant dynamic state feedback controls. One can also evaluate the efficacy of pre-designed dynamic state feedback controllers utilizing this property. Two illustrative examples are given to show the effectiveness of the proposed approach.

L2 control of LPV systems with saturating actuators: Pólya approach

SUMMARY This paper addresses the design problem of , gain-scheduling non-linear state-feedback controller for linear parameter varying (LPV) systems, subjected to actuator saturations and bounded energy disturbances, by using parameter-dependent type Lyapunov functions. The paper provides a systematic procedure to generate a sequence of linear matrix inequality (LMI) type conditions of increasing precision for obtaining a suboptimal state-feedback controller. The presented method utilizes the modified sector condition for formalization of actuator saturation and homogeneous polynomial parameter-dependent representation of LPV systems. Both simulations and experimental studies on an inverted pendulum on a cart system illustrate the benefits of the approach. Copyright © 2011 John Wiley & Sons, Ltd.

Suboptimal Switching State Feedback Control Consistency Analysis for Switched Linear Systems

AbstractThis paper introduces the concept of consistency for continuous-time switched linear systems having the switching function as a primary control signal to be designed. A state feedback switching control strategy is strictly consistent whenever it improves performance compared to the ones of all isolated subsystems. It is proven that a min-type switching strategy is strictly consistent for the classes of H2 and H∞ performance indexes. This property makes clear the importance of switching systems control design in both theoretical and practical application frameworks. Moreover, with this property it is not necessary to assume that all the subsystems are not stabilizable in order to make a switching strategy design problem well posed. The theory is illustrated by means of several academic examples.

H-Infinity State Feedback Control for a Class of Networked Cascade Control Systems With Uncertain Delay

Based on practical industrial process control, a typical configuration for networked cascade control systems (NCCSs) is analyzed. This kind of NCCSs with state feedback controllers, in which the network-induced delay is uncertain and less than a sampling period, is studied. The sufficient condition for the stabilizability of the NCCSs without disturbances is proposed, and the state feedback stabilization control laws are derived via Lyapunov stability theory and linear matrix inequality (LMI) approach. For the NCCSs with disturbances, the criterion of its robust asymptotically stability is derived and the ¿ -suboptimal state feedback <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">¿</sub> control laws are designed. The ¿-optimal state feedback <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">¿</sub> control laws are also put forward by optimizing a set of LMIs. A simulation example of a NCCS for the main steam temperature in a power plant is given to demonstrate the effectiveness of the proposed approaches.

Suboptimal stochastic H-two/H-infinity design with spectrum constraint

This paper studies the problem of suboptimal state-feedback H-two/H-infinity control of stochastic systems with spectrum constraint. Concretely speaking, a mixed suboptimal H-two/H-infinity controller synthesis together with placing the spectrum into a strip region is considered, which is achieved via solving a convex optimization problem.

Cheap Suboptimal Control of an Integral Sliding Mode for Uncertain Systems With Delays

A cheap control problem for a system with state delays and matched uncertainties is studied. First, such a cheap control problem is considered for the nominal linear system associated with the original one. A suboptimal state-feedback composite control for this problem is constructed using a singular perturbation technique. Employing this composite control in the nominal system generates the suboptimal nominal trajectory. Secondly, based on the composite control, an integral sliding mode controller is designed for the original system, providing its robust motion along the suboptimal nominal trajectory. An illustrative numerical example is presented.

Policy Iterations on the Hamilton–Jacobi–Isaacs Equation for $H_{\infty}$ State Feedback Control With Input Saturation

An Hinfin suboptimal state feedback controller for constrained input systems is derived using the Hamilton-Jacobi-Isaacs (HJI) equation of a corresponding zero-sum game that uses a special quasi-norm to encode the constraints on the input. The unique saddle point in feedback strategy form is derived. Using policy iterations on both players, the HJI equation is broken into a sequence of differential equations linear in the cost for which closed-form solutions are easier to obtain. Policy iterations on the disturbance are shown to converge to the available storage function of the associated L2-gain dissipative dynamics. The resulting constrained optimal control feedback strategy has the largest domain of validity within which L2-performance for a given gamma is guaranteed

OPTIMAL CONTROL IN HYBRID SYSTEMS

Necessary conditions for optimality have been established for hybrid systems. Unfortunately, these conditions lead to multi-point boundary value problems and they do not prevent from the combinatorial explosion when no constraints are given on the transitions. Different approaches have been proposed to approximate the solution, such as relaxed dynamic programming, non linear programming and sensitivity functions.In hybrid systems context, the necessary conditions for optimal control are now well known. These conditions mix discrete and continuous classical necessary conditions on the optimal control. The discrete dynamic involves dynamic programming methods whereas between the a priori unknown discrete values of time, optimization of the continuous dynamic is performed using the maximum principle (MP) or Hamilton Jacobi Bellmann equations(HJB). At the switching instants, a set of boundary tranversality necessary conditions ensure a global optimization of the hybrid system.These theoretical conditions were applied to minimum time problem and to linear quadratic optimization.But it is practically very hard to perform such an optimization. The major raison is that discrete dynamic requires evaluating the optimal cost along all branches of the tree of all possible discrete trajectories.Dynamic programming is then used, but the duration between two switchings and the continuous optimization procedure make the task really hard. This makes the complexity increasing and only problems with a poor coupling between continuous and discrete parts can be reasonably solved. Nowadays, it seems obvious that only approximated solutions can be found. Various schemes have been imagined.Recent works have proposed to solve optimal switching problems by using a fixed switching schedule. By switched systems we mean a class of hybrid dynamical systems consisting of a family of continuous (or discrete) time subsystems and a rule (to be determined) that governs the switching between them.The optimization consists then in determining the optimal switching instants and the optimal continuous control assuming the number of switchings and the order of active subsystems already given. Then a nonlinear search method is used to determine the optimal solution.after the calculus of the derivatives of the value function with respect to the switching instants.Relaxed Dynamic programming : a relaxed procedure based on upper and lower bounds of the optimal cost was recently introduced. It proved to give good results for piece-wise affine systems and to obtain a suboptimal state feedback solution in the case of a quadratic criteria Algorithms based on the maximum principle for both multiple controlled and autonomous switchings with fixed schedule have been proposed. The algorithms use the transversality conditions at switching instants. Then, the authors develop a combinational search in order to determine the optimal switching schedule Interesting results on state or output feedback have been given with the regions of the state space where an optimal mode switch should occur.In complement of all the methods resulting from the resolution of the necessary conditions of optimality, we propose to use a multiple-phase multiple-shooting formulation which enables the use of standard constraint nonlinear programming methods. This formulation is applied to hybrid systems with autonomous and controlled switchings and seems to be of interest in practice due to the simplicity of implementation.Sensitivity analysis is the key point of all the methods based on non linear programming. It can also be used to determine limit cycles and the optimal strategy to reach them. All these items are discussed in the plenary session.

Control of the Burgers Equation by a Reduced-Order Approach Using Proper Orthogonal Decomposition

Proper orthogonal decomposition (POD) is a method to derive reduced-order models for dynamical systems. In this paper, POD is utilized to solve open-loop and closed-loop optimal control problems for the Burgers equation. The relative simplicity of the equation allows comparison of POD-based algorithms with numerical results obtained from finite-element discretization of the optimality system. For closed-loop control, suboptimal state feedback strategies are presented.

Direct Method of Constructing H 2-Suboptimal Controllers: Continuous-Time Systems

An algorithm called the 'H/sub 2/ suboptimal state feedback gain sequence' (H/sub 2/ SOSFGS) algorithm is developed. Rather than utilizing a 'perturbation method' which is numerically stiff and computationally prohibitive, the H/sub 2/ SOSFGS algorithm utilizes a direct eigenvalue assignment method to come up with a sequence of H/sub 2/ suboptimal state feedback gains. Also, although the sequence of H/sub 2/ suboptimal state feedback gains constructed by the H/sub 2/ SOSFGS algorithm depends on a parameter /spl epsi/, the construction procedure itself does not require explicitly the value of the parameter /spl epsi/. For a given H/sub 2/ suboptimal state feedback gain, a sequence of observer gains for either a full or a reduced order observer can be constructed by merely dualizing the H/sub 2/ SOSFGS algorithm.