Linear Quadratic Optimal Control Research Articles

Indefinite Linear-Quadratic Optimal Control of Mean-Field Stochastic Differential Equation With Jump Diffusion: An Equivalent Cost Functional Method

Indefinite Linear-Quadratic Optimal Control of Mean-Field Stochastic Differential Equation With Jump Diffusion: An Equivalent Cost Functional Method

Feedback–feedforward signal control with exogenous demand estimation in congested urban road networks

To cope with uncertain traffic patterns and traffic models, traffic-responsive signal control strategies in the literature are designed to be robust to these uncertainties. These robust strategies still require sensing infrastructure to implement traffic-responsiveness. In this paper, we take a novel perspective and show that it is possible to use the already necessary sensing infrastructure to estimate the uncertain quantities in real time. Specifically, resorting to the store-and-forward model, we design a novel network-wide traffic-responsive strategy that estimates the occupancy and exogenous demand in each link, i.e., entering (exiting) vehicle flows at the origins (destinations) of the network or within links, in real time. Borrowing from optimal control theory, we design an optimal linear quadratic control scheme, consisting of a linear feedback term, of the occupancy of the road links, and a feedforward component, which accounts for the varying exogenous vehicle load on the network. Thereby, the resulting control scheme is a simple feedback–feedforward controller, which is fed with occupancy and exogenous demand estimates, and is suitable for real-time implementation. Numerical simulations for the urban traffic network of Chania, Greece, show that, for realistic surges in the exogenous demand, the proposed solution significantly outperforms tried-and-tested solutions that ignore the exogenous demand.

Be greedy and learn: efficient and certified algorithms for parametrized optimal control problems

We consider parametrized linear-quadratic optimal control problems and provide their online-efficient solutions by combining greedy reduced basis methods and machine learning algorithms. To this end, we first extend the greedy control algorithm, which builds a reduced basis for the manifold of optimal final time adjoint states, to the setting where the objective functional consists of a penalty term measuring the deviation from a desired state and a term describing the control energy. Afterwards, we apply machine learning surrogates to accelerate the online evaluation of the reduced model. The error estimates proven for the greedy procedure are further transferred to the machine learning models and thus allow for efficient a posteriori error certification. We discuss the computational costs of all considered methods in detail and show by means of two numerical examples the tremendous potential of the proposed methodology.

Optimization design of unmanned mining truck path tracking controller based on road adhesion coefficient estimation

Abstract In response to the issues of inaccurate tracking precision caused by continuous turning and variable road adhesion coefficients on complex roads in mining areas for unmanned mining trucks, this paper adopts unscented Kalman filter (UKF) to estimate the road adhesion coefficient. Based on this, an adaptive preview feedforward linear quadratic regulator (LQR) control algorithm is designed using genetic algorithm, feedforward control, and linear quadratic optimal path tracking control methods. Firstly, a nine degree of freedom vehicle dynamics model and a two degree of freedom vehicle lateral dynamics model are established, and the Dugoff model to calculate tire forces is introduced; secondly, utilizing UKF to estimate the road adhesion coefficient, and leveraging active disturbance rejection control for rapid tracking of road curvature, an optimized design of the LQR controller is carried out. Then, the effectiveness of the designed controller was verified through Trucksim/Simulink joint simulation testing. Finally, the hardware in the loop test results showed that the designed controller had good tracking accuracy and strong adaptability in complex road and special working conditions in mining areas.

Synthesis of a Regulator for a Linear-Quadratic Optimal Control Problem

Synthesis of a Regulator for a Linear-Quadratic Optimal Control Problem

Linear quadratic optimal control of stochastic 2-D Roesser models

This paper investigates the linear quadratic (LQ) optimal control problem for the stochastic two-dimensional (2-D) systems governed by Roesser models with multiplicative noise. The main contribution is to give the necessary and sufficient optimality condition by proposing a set of novel forward and backward stochastic partial difference equations (FBSPDE), and to further present the explicitly optimal feedback control laws on the finite horizon and on the infinite horizon based on the Riccati-like difference equations and the algebraic equation, respectively. Several numerical simulations are provided to illustrate the performance of the designed controllers.

System transformation and model-free value iteration algorithms for continuous-time linear quadratic stochastic optimal control problems

In this paper, we investigate a continuous-time linear quadratic stochastic optimal control (LQSOC) problem in an infinite horizon, where diffusion and drift terms of the corresponding stochastic system depend on both state and control variables. In light of the stochastic control theory, this LQSOC problem is reduced to solving a generalised algebraic Riccati equation (GARE). With the help of an existing model-based value iteration (VI) algorithm, we propose two data-driven VI algorithms to solve the GARE. The first one relies on transforming the stochastic system into a deterministic control system first and then solving the LQSOC problem by the data of the deterministic system. Consequently, this algorithm does not need the information of two system coefficients and has a lower algorithm complexity. The second algorithm directly uses the data generated by the stochastic system, and thus it circumvents the requirement of all system coefficients. We also provide convergence proofs of these two data-driven algorithms and validate these algorithms through two simulation examples.

Research on semi-active vibration reduction of offshore wind turbine structure combined eddy current theory with tuned mass dampers

At present, the offshore wind power gradually shows a development trend towards the profound ocean with characteristic of larger capacity, higher tower and longer blades. Therefore, the offshore wind turbine (OWT) is more vulnerable to extreme loads and may lead to the negative influence of structural vibration safety. In this research, a new semi-active eddy current tuned mass damper (SEC-TMD) was proposed for the vibration control of OWT structures and its vibration reduction effect under multiple kinds of dynamic loads was also discussed. Firstly, taking one actual 3.0 MW OWT supported by bucket foundation as object, the SEC-TMD was designed based on the linear quadratic optimal control (LQR) algorithm and bounded Hrovat optimal control algorithm in order to obtain the optimal control force of structural vibration systems. The relationship between damping coefficient and magnetic conductivity spacing was further fitted to achieve variable damping control of SEC-TMD. Secondly, the theoretical model of 3.0 MW OWT structure installed with SEC-TMD was established and the effectiveness of the established theoretical model was verified through the comparison on modal analysis between the theoretical calculations and measured data identification. Finally, the vibration reduction effect of SEC-TMD on the measured OWT structure under wind-wave, seismic and vibration amplification conditions was analyzed and compared with the results calculated from the other passive dampers including EC-TMD and TLCD. It is shown that the maximum reduction percentage of peak value for vibration displacement and acceleration on the tower top can respectively reach 31.93 %, 44.11 %, 30.20 % and 54.57 %, 33.94 %, 31.75 % under three above conditions. Comparing with EC-TMD and TLCD, the better vibration reduction effect indicates that the proposed SEC-TMD has good engineering application value for the vibration suppression of OWT.

Human-in-the-Loop Behavior Modeling via an Integral Concurrent Adaptive Inverse Reinforcement Learning.

One goal of artificial intelligence (AI) research is to teach machines how to learn from humans, such that they can perform a certain task in a natural human-like way. In this article, an online adaptive inverse reinforcement learning (IRL) approach to human behavior modeling is proposed to enhance machine intelligence for a class of linear human-in-the-loop (HiTL) systems using the state data only, where the human behavior is described by a linear quadratic optimal control model with an unknown weighting matrix for the quadratic cost function. First, an integral concurrent adaptive law is developed to learn the human feedback gain matrix online using the demonstrated state data only, which removes the persistent excitation (PE) conditions required by traditional adaptive estimation approaches and thus is more in line with real applications. Then, with the learned feedback gain matrix, the IRL problem is formulated as a linear matrix inequality (LMI) optimization problem, which can be efficiently solved to retrieve the weighting matrix of the human cost function. Finally, a simulation example is provided to illustrate the effectiveness of the proposed approach.

Finite-horizon and infinite-horizon linear quadratic optimal control problems: A data-driven Euler scheme

In this paper, we devote our focus to studying finite-horizon and infinite-horizon linear quadratic optimal control (LQOC) problems in continuous time. We focus on algorithms for approximating optimal controls of these two problems without the knowledge of all system coefficients. Firstly, we transform the backward differential Riccati equation (DRE) corresponding to the finite-horizon LQOC problem into a forward ordinary differential equation. Then, we discretize the forward equation by the Euler method. Subsequently, we use the collected state and input data to calculate all discretized points of the forward equation. At the same time, we obtain the optimal control of the finite-horizon LQOC problem. Moreover, we approximate the optimal control of the infinite-horizon LQOC problem by virtue of an asymptotic behavior between the DRE and an algebraic Riccati equation (ARE) arising in the infinite-horizon LQOC problem. Finally, we validate the obtained results via two practically motivated examples.

Design of reduced-order observer based steady state optimal linear-quadratic feedback controllers

In this paper we formulate and solve a problem encountered in engineering applications when a linear-quadratic (LQ) optimal feedback controller uses state estimates obtained via a reduced-order observer. Due to the use of state estimates instead of the actual state variables, the optimal quadratic performance is degraded in a pretty complex manner. In the paper, we show how to find the exact formula for the optimal performance degradation (optimal performance loss) in linear time invariant systems for the steady state case (infinite horizon optimization problem). The optimal performance loss is obtained in terms of solution of a reduced-order algebraic Lyapunov equation whose dimension is equal to the dimension of the reduced-order observer. The quantities that impact the performance criterion loss are identified. Practical examples (an inverted pendulum on a cart and an aircraft) show that the optimal performance loss can be very significant in some applications, and even very high in the presented linear-quadratic optimal control of an inverted pendulum problem. We have shown, using the derived formula, how the optimal performance loss can be considerably reduced by properly choosing the reduced-order observer initial conditions via the least square method.

Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the Related Hamiltonian Systems

Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the Related Hamiltonian Systems

Linear quadratic optimal control problems of infinite‐dimensional mean‐field type with jumps

AbstractIn this paper, we formulate and investigate a framework for the theory of the linear quadratic optimal control problem (LQ problem) for infinite‐dimensional mean‐field stochastic evolution systems with jumps. We ensure the well‐posedness of the investigated problems by establishing the existence, uniqueness, and a priori estimates for mild solutions to general infinite‐dimensional mean‐field forward stochastic evolution equations (MF‐SEE) and mean‐field backward stochastic evolution equations (MF‐BSEE) with jumps. Leveraging the Yosida approximation theory, we establish a dual theory between MF‐SEE and MF‐BSEE with jumps, overcoming the challenge posed by the inapplicability of Itô's formula in the context of mild solutions. Our main results regarding the existence and uniqueness of the optimal control, along with corresponding dual characterizations and state feedback representations, are obtained through convex analysis techniques, our established dual theory, and decoupling methods.

Design of Static Output Feedback Suspension Controllers for Ride Comfort Improvement and Motion Sickness Reduction

This paper presents a method to design a static output feedback active suspension controller for ride comfort improvement and motion sickness reduction in a real vehicle system. Full-state feedback controller has shown good performance for active suspension control. However, it requires a lot of states to be measured, which is very difficult in real vehicles. To avoid this problem, a static output feedback (SOF) controller is adopted in this paper. This controller requires only three sensor outputs, vertical velocity, roll and pitch rates, which are relatively easy to measure in real vehicles. Three types of SOF controller are proposed and optimized with linear quadratic optimal control and the simulation optimization method. Two of these controllers have only three gains to be tuned, which are much smaller than those of full-state feedback. To validate the performance of the proposed SOF controllers, a simulation is carried out on a vehicle simulation package. From the results, the proposed SOF controllers are quite good at improving ride comfort and reducing motion sickness.

Linear quadratic optimal control turnpike in finite and infinite dimension: Two-term expansion of the value function

In this paper, we consider a linear quadratic (LQ) optimal control problem in both finite and infinite dimensions. We derive an asymptotic expansion of the value function as the fixed time horizon T tends to infinity. The leading term in this expansion, proportional to T, corresponds to the optimal value attained through the classical turnpike theory in the associated static problem. The remaining terms are associated with optimal stabilization problems towards the turnpike.

Design of Active Suspension Controller for Ride Comfort Enhancement and Motion Sickness Mitigation

This paper presents a method for designing an active suspension controller for ride comfort enhancement and motion sickness mitigation. For this, it is necessary to design an active suspension controller, which aims to reduce the vertical acceleration and pitch rate of a sprung mass in a vehicle. A half-car vehicle model was selected. For the controller design, a static output feedback (SOF) control was selected instead of a full-state feedback control because it is hard to measure all state variables in real vehicles. With the available signals, three types of SOF controller were proposed. To determine the gains of the SOF controllers, a linear quadratic optimal control methodology and a simulation-based optimization method were adopted. To validate the proposed method, a simulation was carried out using vehicle simulation software. The simulation results show that the proposed method is quite effective for ride comfort enhancement and motion sickness mitigation.

Decentralized stochastic linear-quadratic optimal control with risk constraint and partial observation

This paper addresses a risk-constrained decentralized stochastic linear-quadratic optimal control problem with one remote controller and one local controller, where the risk constraint is posed on the cumulative state weighted variance in order to reduce the oscillation of system trajectory. In this model, local controller can only partially observe the system’s state, and sends the estimate of state to remote controller through an unreliable channel, whereas the channel from remote controller to local controller is perfect; here, the unreliable channel means the Bernoulli channel, that is, when data is transmitted on this channel, there is a certain probability of failure. For the considered constrained optimization problem with Bernoulli channel, we first punish the risk constraint into cost function through Lagrange multiplier method, and the resulting augmented cost function will include a quadratic mean-field term of state. In the sequel, for any but fixed multiplier, explicit solutions to finite-horizon and infinite-horizon mean-field decentralized linear-quadratic problems are derived together with necessary and sufficient condition on the mean-square stability of optimal system. Then, approach to find the optimal Lagrange multiplier is presented based on bisection method. Finally, two numerical examples are given to show the efficiency of the obtained results.

Optimal control for both forward and backward discrete-time systems

Forward discrete-time systems use past information to update the current state, while backward discrete-time systems use future information to update the current state. This study focuses on optimal control problems within the context of forward and backward discrete-time systems. We begin by investigating a general optimal control problem for both forward and backward discrete-time systems. Leveraging the inherent properties of these systems and the Bellman optimality principle, we derive recursive equations as a means to solve such optimal control problems. Using these recursive equations, we obtain analytical expressions for both the optimal controls and optimal values of bang–bang and linear quadratic optimal control problems. Finally, we present a numerical example and an industrial wastewater treatment problem to illustrate and demonstrate our findings.

Pointwise Error Estimates of Numerical Solutions to Linear Quadratic Optimal Control Problems

An auxiliary optimal control problem is formulated that provides with its unique solution, a continuous representation of the global error of a numerical approximation to the solution of a linear quadratic optimal control problem. The resulting error functions are characterized as the unique solutions to an optimality system that is reformulated as a boundary value problem. With this formulation, reliable pointwise error estimates can be generated utilizing well-established techniques of defect control. A novel algorithm based on defect correction and defect control is presented that generates pointwise approximations to the global error of numerical optimal control solutions on a uniform grid. It is proven and numerically validated that this algorithm can generate pointwise estimates that approximate the true global error with a prescribed accuracy.

Stochastic LQ optimal control for Markov jumping systems with multiplicative noise using reinforcement learning

The stochastic linear quadratic (LQ) problem of discrete-time linear Markov jumping systems with multiplicative noise is investigated in this paper. Two reinforcement learning algorithms, one model-based and one model-free, are designed. The algorithms can be readily adapted to handle nonlinear cases by employing an appropriate function approximator. In the linear case, the network structure is designed to enable the problem to be solved without succumbing to modeling errors. For the model-free algorithm, an approach based on moving averages is proposed to reduce estimation variance. A proof of convergence is also provided. Finally, the effectiveness of the proposed algorithms is demonstrated through a numerical example.