Adaptive Linear Quadratic Control Research Articles

Adaptive Linear Quadratic Control for Stochastic Discrete-Time Linear Systems With Unmeasurable Multiplicative and Additive Noises

Adaptive Linear Quadratic Control for Stochastic Discrete-Time Linear Systems With Unmeasurable Multiplicative and Additive Noises

An averaging approach for nearly-optimal control of multi-zone buildings

Controlling Heating, Ventilation and Air-Conditioning (HVAC) in a way that is both optimal (energy efficient) and adaptive (able to cope with uncertainties) is a recognized challenge in the building sector. This work presents a simple modelling approach to HVAC operation aiming for adaptive and optimal control. The proposed framework exhibits several advantages amenable for implementation in energy management systems: minimum number of parameters to be estimated; preserving the distributed nature of multiple zones in the estimation process; easiness of imposing positivity and controllability constraints on the estimated coefficients. Optimality is addressed using an adaptive linear quadratic control philosophy. Comparisons with Proportional-Integral-Derivative (PID) and non-adaptive controllers show the capability of the proposed framework of coping with uncertainty while delivering nearly optimal controls.

Optimizing Signal Timing Control for Large Urban Traffic Networks Using an Adaptive Linear Quadratic Regulator Control Strategy

Traffic signal control is important for intersection safety and efficiency. However, most traffic signal control methods are designed for individual intersections or corridors. Although some adaptive control systems have been developed, the methods used are often proprietary and not published, making it difficult to evaluate their effectiveness. This study proposes an adaptive multi-input and multi-output traffic signal control method that not only can improve network-wide traffic operations in terms of reduced traffic delay and energy consumption, but also is more computationally feasible than existing centralized signal control methods. Considering intersection interactions, a linear dynamic traffic system model was built and adaptively updated to reflect how the signal control input of each intersection affects network-wide vehicle travel delay. Based on the system model, an adaptive linear-quadratic regulator (LQR) was designed to minimize both traffic delay and incremental changes in the control input. The proposed control method was evaluated in a microscopic traffic simulation environment with a 35-intersection network of Bellevue City, Washington. Simulation results show that the proposed method had shorter average traffic delays in the network when compared with the traffic delays controlled by the state-of-the-art max-pressure, self-organizing traffic lights, and independent deep Q network methods.

Adaptive Control Strategies for Urban Network Traffic via a Decentralized Approach With User-Optimal Routing

This paper presents an adaptive linear quadratic optimal traffic control system. The control strategies are solved via a decentralized approach and complemented with a user-optimal network traffic router. The user-optimal routing algorithm assists drivers respond to prevailing traffic state and control settings and seek the quickest route toward their destinations. The proposed control system is implemented and tested over different scenario settings including a real life scenario in Central London, UK. The study reveals that the proposed system could coverage to a performance similar to its centralized counterpart with the routing algorithm even under congested conditions. This highlights the potential of decentralized control with effective travel guidance in cooperative traffic management.

Implementation of an Adaptive Linear Quadratic Control Based on Cost-Biased Approach

Implementation of an Adaptive Linear Quadratic Control Based on Cost-Biased Approach

Distributed Adaptive Linear Quadratic Control using Distributed Reinforcement Learning

Abstract In this paper distributed adaptive linear quadratic control of discrete-time linear large-scale systems with unknown dynamics using distributed reinforcement learning is studied. Linear quadratic control based on dynamic programming (specifically policy iteration) and adaptive linear quadratic control based on reinforcement learning (especially Q learning) are reviewed first. Then distributed adaptive linear quadratic control is addressed. Two Q functions exploiting the quadratic structure of the value function and leading to a decentralized and a distributed policy are proposed and a decentralized as well as a distributed Q learning algorithm are presented. Finally the concepts are evaluated in a simulation study. The simulation results indicate that the distributed policy is near-optimal.

Adaptive Actuator Failure Compensation for Possibly Nonminimum-Phase Systems Using Control Separation Based LQ Design

This paper develops an adaptive actuator failure compensation control scheme for discrete-time and possibly nonminimum-phase systems with unknown actuator failures of characterizations that first, some unknown system inputs are stuck at some unknown fixed or varying values at unknown time instants, and second, the actuator failures as system disturbances are not matched by other inputs. Such a failure compensation control problem is solved by using a new control separation design based on a new linear quadratic (LQ) control formulation: One control component is used for disturbance rejection and actuator failure compensation, and one component is used for closed-loop stability and regulation, to achieve the desired control objective. The nominal LQ control solution uses a new cost function to derive the two-component control signal, which has improved performance over a traditional LQ control design. The adaptive actuator failure compensation control problem is solved using a stable adaptive law for parameter estimation, to guarantee the system stability and output regulation performances in the presence of plant and actuator failure uncertainties. Simulation results are presented to verify the satisfactory performance of the proposed adaptive LQ control scheme.

Robust Nonlinear Adaptive Control of a “Boost” Converter via Algebraic Parameter Identification

This paper presents the design of a robust nonlinear adaptive controller for trajectory tracking maneuvers of the nonminimum phase output voltage on a dc-to-dc “boost” power converter with uncertain time-varying parameters. The unknown parameter variations concern both the resistive load and the input voltage supply value. A generalized proportional-integral (GPI) indirect control, exploiting the flatness property of the system, performs fast adaptations on the feedback controller and on the desired output reference trajectory owing to a fast online algebraic parameter identification procedure. The required updating of the algebraic parameter identification process is periodically triggered to cope with the time variations of the unknown plant parameters. The proposed control scheme is shown to be quite effective for handling the significant plant uncertainties when tested through experiments on a laboratory prototype. An adaptive linear quadratic regulator optimal control based on feedback linearization was designed in order to compare its performance against that of the proposed GPI adaptive scheme. An integral square error index was used for the evaluation.

Optimal Choice of Weighting Factors in Adaptive Linear Quadratic Control

Most of the processes in the industry have nonlinear behavior. Control of such processes with conventional control methods could lead to unstable, suboptimal, etc., results. On the other hand, the adaptive control is a technique widely used for controlling of nonlinear systems. The approach here is based on the recursive identification of the external linear model as a linear representation of the originally nonlinear system. The controller then reacts to the change of the working point or disturbances which could occur by the change of the parameters, structure, etc. The polynomial synthesis together with the linear quadratic (LQ) approach is employed here for the controller synthesis. These techniques satisfy basic control requirements such as the stability, the reference signal tracking and the disturbance attenuation. Resulted controller could be tuned with the choice of weighting factors in LQ approach. This work investigates the effect of these factors on control results. Proposed methods are tested on the mathematical model of the isothermal continuous stirred-tank reactor and simulated results are also verified on the real model of the continuous stirred tank reactor.

Hybrid Fuzzy Logic and Adaptive LQR Controller for Swing-up, Positioning and Stabilization of Inverted Pendulum

This paper presents the design and practical implementation of a hybrid fuzzy logic and adaptive linear–quadratic controller (LQR) for a real inverted short pendulum system. We present an extended swing-up approach using fuzzy controller and then discuss an adaptive LQR realization which takes into account nonlinearities while passing the transient process to the upward position of the short pendulum which is mounted on a cart. So long as the cart’s configuration space is restricted by boundary conditions the controller also solves the positioning task, during which the cart returns to the centre of cart’s configuration space. We also discuss the practical realization of such controller logic, embedded into 32-bit microcontroller with the algorithm reaction of 1 ms. DOI: http://dx.doi.org/10.5755/j01.eee.20.3.6675

Double-stepped adaptive control for hybrid systems with unknown Markov jumps and stochastic noises

This paper is concerned with the sampled-data based adaptive linear quadratic (LQ) control of hybrid systems with both unmeasurable Markov jump processes and stochastic noises. By the least matching error estimation algorithm, parameter estimates are presented. By a double-step (DS) sampling approach and the certainty equivalence principle, a sampled-data based adaptive LQ control is designed. The DS-approach is characterized by a comparatively large estimation step for parameter estimation and a sufficient small control step for control updating. Under mild conditions, the closed-loop system is shown to be stable. It is found that the key factor determining the perfor- mance index is the estimation step rather than the control step. When the estimation step becomes too small, the system performance will become worse. When the estimation step is fixed, the system performance can indeed be improved by reducing the control step, but cannot reach the optimal value. The index difference between the sampled-data based adaptive LQ control and the conventional LQ optimal control is asymptotically bounded by a constant depending on the estimation step and the priori information of the parameter set.

Adaptive Linear Quadratic Control for High Dynamical and Comfortable Behavior of a Heavy Truck

Adaptive Linear Quadratic Control for High Dynamical and Comfortable Behavior of a Heavy Truck

Adaptive sampled-data based linear quadratic optimal control of stochastic systems

The problem of sampled-data (SD) based adaptive linear quadratic (LQ) optimal control is considered for linear stochastic continuous-time systems with unknown parameters and disturbances. To overcome the difficulties caused by the unknown parameters and incompleteness of the state information, and to probe into the influence of sample size on system performance, a cost-biased parameter estimator and an adaptive control design method are presented. Under the assumption that the unknown parameter belongs to a known finite set, some sufficient conditions ensuring the convergence of the parameter estimate are obtained. It is shown that when the sample step size is small, the SD-based adaptive control is LQ optimal for the corresponding discretized system, and sub-optimal compared with that of the case where the parameter is known and the information is complete.

Optimality for the linear quadratic non‐Gaussian problem via the asymmetric Kalman filter

AbstractIn the linear non‐Gaussian case, the classical solution of the linear quadratic Gaussian (LQG) control problem is known to provide the best solution in the class of linear transformations of the plant output if optimality refers to classical least‐squares minimization criteria. In this paper, the adaptive linear quadratic control problem is solved with optimality based on asymmetric least‐squares approach, which includes least‐squares criteria as a special case. Our main result gives explicit solutions for this optimal quadratic control problem for partially observable dynamic linear systems with asymmetric observation errors. The main difficulty is to find the optimal state estimate. For this purpose, an asymmetric version of the Kalman filter based on asymmetric least‐squares estimation is used. We illustrate the applicability of our approach with numerical results. Copyright © 2004 John Wiley & Sons, Ltd.

Stabilization of stochastic systems with hidden Markovian jumps

This paper considers the adaptive control of discrete-time hybrid stochastic systems with unknown randomly jumping parameters described by a finite-state hidden Markov chain. An intuitive yet longstanding conjecture in this area is that such hybrid systems can be adaptively stabilized whenever the rate of transition of the hidden Markov chain is small enough. This paper provides a rigorous positive answer to this conjecture by establishing the global stability of a gradient-algorithm-based adaptive linear-quadratic control.

Robust Adaptive Controller with Fine Grain Parallelism

Two issues associated with an adaptive linear quadratic controller are addressed in the paper: numerical robustness of the adaptive estimator with respect to low excitation in the closed control loop and rather high computational complexity of the controller.The algorithm presented in the paper uses The estimator used in the paper is based on the inverse-updated square-root recursive least squares (SR-RLS) identification algorithm (Moonen and McWhirter 1993). To increase numerical robustness of the algorithm for weak excitation, the regularized exponential forgetting is used. The multi-step control design uses an autoregression system model with exogenous input (ARX model).Computational complexity of the resulting algorithm is reduced by using the systolic paradigm for its implementation.

Stochastic linear quadratic adaptive control for continuous-time first-order systems

In this paper, we discuss the linear quadratic (LQ) adaptive control problem for the following continuous-time first-order scalar stochastic system: d x t = ax t d t + bu t d t + c d w t , with cost function min lim sup J t ( u), u∈ U t→∞ where J t(U)= 1 t ∫ 0 t (q1X 2 s + q2U 2 s)ds, q1⩽0,>0 Based on the self-convergence property of continuous-time weighted least-squares (CWLS) algorithm [9] and the similar parameter modification method [13], the stability of the closed-loop system is achieved without invoking any excitations. Here self-convergence means that the convergence is automatic in the sense that no excitation conditions on the signals or measurements is needed. It is a terminology from [10]. The strong consistency of CWLS and the optimality of the adaptive control are established by incorporating with diminishing excitations.

Microbial adaptation, process performance and a suggested control strategy in a pre-denitrifying system with ethanol dosage

Biological nitrogen removal in activated sludge processes is dependent on sufficient supplies of easily metabolized carbon compounds for the denitrifying bacterial population. An external carbon source can increase denitrification rates and compensate for deficiencies in the influent C/N ratio. Plant performance and microbial adaptation were studied in a pre-denitrifying pilot-scale activated sludge plant with and without ethanol. Total nitrogen removal efficiency was 67 and 35% for the ethanol and reference line, respectively. The process responded rapidly to ethanol but one sludge age was necessary for full bacterial adaptation. An initial rapid increase suggests enzyme induction rather than alterations in bacterial species composition. Increased enzyme activity was explained by an increase in turn-over rate of biomass. Low effluent nitrate concentration was a result of the simultaneous use of influent COD and ethanol. Fluctuations in influent COD did not affect denitrification capacity with ethanol. Sludge settling properties were moderately better in the process without ethanol addition. An automatic control strategy for carbon dosage using feedforward from influent carbon and nitrate in the recirculated flow was simulated. Simulations with an adaptive linear quadratic controller demonstrated that the desired nitrate concentration at the end of the anoxic zone could be maintained despite relatively large disturbances.

Microbial adaptation, process performance and a suggested control strategy in a pre-denitrifying system with ethanol dosage

Biological nitrogen removal in activated sludge processes is dependent on sufficient supplies of easily metabolized carbon compounds for the denitrifying bacterial population. An external carbon source can increase denitrification rates and compensate for deficiencies in the influent C/N ratio. Plant performance and microbial adaptation were studied in a pre-denitrifying pilot-scale activated sludge plant with and without ethanol. Total nitrogen removal efficiency was 67 and 35% for the ethanol and reference line, respectively. The process responded rapidly to ethanol but one sludge age was necessary for full bacterial adaptation. An initial rapid increase suggests enzyme induction rather than alterations in bacterial species composition. Increased enzyme activity was explained by an increase in turn-over rate of biomass. Low effluent nitrate concentration was a result of the simultaneous use of influent COD and ethanol. Fluctuations in influent COD did not affect denitrification capacity with ethanol. Sludge settling properties were moderately better in the process without ethanol addition. An automatic control strategy for carbon dosage using feedforward from influent carbon and nitrate in the recirculated flow was simulated. Simulations with an adaptive linear quadratic controller demonstrated that the desired nitrate concentration at the end of the anoxic zone could be maintained despite relatively large disturbances.

On the Linear Quadratic Adaptive Control for the Design of Missile Autopilots

This paper deals with an adaptive control application involving a skid-to-turn (STT) missile. More specifically, a linear quadratic robust adaptive controller is used to deal with the yaw channel control problem. The linear quadratic control design is mainly motivated by its popular stability margins, while the robust parameter adaptation is expected to handle the missile model parameter uncertainties. The control design as well as the parameier estimation are derived in the discrete-time domain using the δ operator formulation. The stale variables as well as the stability margin are shown to demonstrate how the adaptive controller outperforms its underlying fixed parameter controller