Linear Quadratic Optimal Control Research Articles

Design of Static Output Feedback Suspension Controllers for Ride Comfort Improvement and Motion Sickness Reduction

This paper presents a method to design a static output feedback active suspension controller for ride comfort improvement and motion sickness reduction in a real vehicle system. Full-state feedback controller has shown good performance for active suspension control. However, it requires a lot of states to be measured, which is very difficult in real vehicles. To avoid this problem, a static output feedback (SOF) controller is adopted in this paper. This controller requires only three sensor outputs, vertical velocity, roll and pitch rates, which are relatively easy to measure in real vehicles. Three types of SOF controller are proposed and optimized with linear quadratic optimal control and the simulation optimization method. Two of these controllers have only three gains to be tuned, which are much smaller than those of full-state feedback. To validate the performance of the proposed SOF controllers, a simulation is carried out on a vehicle simulation package. From the results, the proposed SOF controllers are quite good at improving ride comfort and reducing motion sickness.

Linear quadratic optimal control turnpike in finite and infinite dimension: Two-term expansion of the value function

In this paper, we consider a linear quadratic (LQ) optimal control problem in both finite and infinite dimensions. We derive an asymptotic expansion of the value function as the fixed time horizon T tends to infinity. The leading term in this expansion, proportional to T, corresponds to the optimal value attained through the classical turnpike theory in the associated static problem. The remaining terms are associated with optimal stabilization problems towards the turnpike.

Design of Active Suspension Controller for Ride Comfort Enhancement and Motion Sickness Mitigation

This paper presents a method for designing an active suspension controller for ride comfort enhancement and motion sickness mitigation. For this, it is necessary to design an active suspension controller, which aims to reduce the vertical acceleration and pitch rate of a sprung mass in a vehicle. A half-car vehicle model was selected. For the controller design, a static output feedback (SOF) control was selected instead of a full-state feedback control because it is hard to measure all state variables in real vehicles. With the available signals, three types of SOF controller were proposed. To determine the gains of the SOF controllers, a linear quadratic optimal control methodology and a simulation-based optimization method were adopted. To validate the proposed method, a simulation was carried out using vehicle simulation software. The simulation results show that the proposed method is quite effective for ride comfort enhancement and motion sickness mitigation.

Decentralized stochastic linear-quadratic optimal control with risk constraint and partial observation

This paper addresses a risk-constrained decentralized stochastic linear-quadratic optimal control problem with one remote controller and one local controller, where the risk constraint is posed on the cumulative state weighted variance in order to reduce the oscillation of system trajectory. In this model, local controller can only partially observe the system’s state, and sends the estimate of state to remote controller through an unreliable channel, whereas the channel from remote controller to local controller is perfect; here, the unreliable channel means the Bernoulli channel, that is, when data is transmitted on this channel, there is a certain probability of failure. For the considered constrained optimization problem with Bernoulli channel, we first punish the risk constraint into cost function through Lagrange multiplier method, and the resulting augmented cost function will include a quadratic mean-field term of state. In the sequel, for any but fixed multiplier, explicit solutions to finite-horizon and infinite-horizon mean-field decentralized linear-quadratic problems are derived together with necessary and sufficient condition on the mean-square stability of optimal system. Then, approach to find the optimal Lagrange multiplier is presented based on bisection method. Finally, two numerical examples are given to show the efficiency of the obtained results.

Optimal control for both forward and backward discrete-time systems

Forward discrete-time systems use past information to update the current state, while backward discrete-time systems use future information to update the current state. This study focuses on optimal control problems within the context of forward and backward discrete-time systems. We begin by investigating a general optimal control problem for both forward and backward discrete-time systems. Leveraging the inherent properties of these systems and the Bellman optimality principle, we derive recursive equations as a means to solve such optimal control problems. Using these recursive equations, we obtain analytical expressions for both the optimal controls and optimal values of bang–bang and linear quadratic optimal control problems. Finally, we present a numerical example and an industrial wastewater treatment problem to illustrate and demonstrate our findings.

Pointwise Error Estimates of Numerical Solutions to Linear Quadratic Optimal Control Problems

An auxiliary optimal control problem is formulated that provides with its unique solution, a continuous representation of the global error of a numerical approximation to the solution of a linear quadratic optimal control problem. The resulting error functions are characterized as the unique solutions to an optimality system that is reformulated as a boundary value problem. With this formulation, reliable pointwise error estimates can be generated utilizing well-established techniques of defect control. A novel algorithm based on defect correction and defect control is presented that generates pointwise approximations to the global error of numerical optimal control solutions on a uniform grid. It is proven and numerically validated that this algorithm can generate pointwise estimates that approximate the true global error with a prescribed accuracy.

Stochastic LQ optimal control for Markov jumping systems with multiplicative noise using reinforcement learning

The stochastic linear quadratic (LQ) problem of discrete-time linear Markov jumping systems with multiplicative noise is investigated in this paper. Two reinforcement learning algorithms, one model-based and one model-free, are designed. The algorithms can be readily adapted to handle nonlinear cases by employing an appropriate function approximator. In the linear case, the network structure is designed to enable the problem to be solved without succumbing to modeling errors. For the model-free algorithm, an approach based on moving averages is proposed to reduce estimation variance. A proof of convergence is also provided. Finally, the effectiveness of the proposed algorithms is demonstrated through a numerical example.

Turnpike Properties for Mean-Field Linear-Quadratic Optimal Control Problems

Turnpike Properties for Mean-Field Linear-Quadratic Optimal Control Problems

A unified study of the redundant control input problem with different conditions in the stabilizable case

This paper investigates the redundant control input problem in the sense of linear quadratic regulator optimal control. Increasing redundant control inputs leads to a decrease in the quadratic performance index. However, in some cases, this also increases the norm of the feedback matrix, resulting in actuator saturation. To solve the problem, a comparative analysis of the latest research results on the redundant control input problem under different conditions is first presented. Secondly, new tighter upper bounds of the positive (semi-)definite solution of the continuous algebraic Riccati equation (CARE) corresponding to the redundant control input problem are studied uniformly in the stabilizable case. Thirdly, based on them, a class of sufficient conditions to guarantee that the norm of the feedback matrix decreases is presented. Finally, numerical experiments demonstrate the effectiveness and superiority of our conclusions. Meanwhile, a simulation experiment demonstrates that systems with redundant control inputs can achieve good control performance.

Forward–backward stochastic evolution equations in infinite dimensions and application to LQ optimal control problems

This paper focuses on the study of forward–backward stochastic evolution equations (FBSEEs), which are a class of nonlinear fully coupled forward–backward stochastic differential equations (FBSDEs), in infinite dimensions. Drawing inspiration from various linear-quadratic (LQ) optimal control problems, we apply a set of domination-monotonicity conditions that are more relaxed compared to general conditions. Within this framework, we employ the method of continuation to establish the unique solvability result and provide a pair of solution estimates. Notably, to address the challenges posed by the infinite-dimensional setting, we introduce a class of approximating equations, as Itô’s formula is not directly applicable. Conversely, these results find application in various LQ problems, where the stochastic Hamiltonian systems precisely correspond to the FBSEEs satisfying the aforementioned domination-monotonicity conditions. Consequently, by solving the corresponding stochastic Hamiltonian systems, we can obtain explicit expressions for the optimal controls.

Optimal Control Strategy of Electro-hydraulic Position Servo System Using Genetic Algorithm

Background: The optimal control strategy has been widely used in electro-hydraulic position servo systems to achieve high-precision position tracking. However, the difficulty of selecting the weighted matrices in optimal control often leads to poor tracking accuracy. Objective: This patent proposes an optimal control strategy using a genetic algorithm to improve the tracking accuracy of the electro-hydraulic servo system. Methods: The patent first established the system state equation of the valve-controlled asymmetric cylinder. Secondly, based on linear quadratic optimal control theory and genetic algorithm, an optimal control strategy using a genetic algorithm was proposed. Finally, the simulation and experimental results showed that the designed controller has high position tracking accuracy . Results: The optimal controller using a genetic algorithm was designed using Matlab/Simulink, and the effectiveness of the controller was verified through simulation. Additionally, experimental results showed that the proposed optimal control controller using a genetic algorithm had higher tracking accuracy than the proportional-integral-derivative controller and traditional backstepping controller for a given reference signal. Conclusion: The control technology of the optimal controller using a genetic algorithm was found to be superior to proportional-integral-derivative and traditional backstepping controllers, and the tracking error of the linear quadratic regulator controller was reported to be relatively small. This demonstrated the effectiveness of the optimal control strategy using a genetic algorithm in this patent.

Linear Quadratic Optimal Control for Systems Governed by First-Order Hyperbolic Partial Differential Equations

Linear Quadratic Optimal Control for Systems Governed by First-Order Hyperbolic Partial Differential Equations

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

Value iteration for LQR control of unknown stochastic-parameter linear systems

This paper focuses on the linear-quadratic optimal control problem for unknown stochastic-parameter linear systems using reinforcement learning methods. Based on the second moments of random system matrices, a model-based value iteration algorithm is proposed to solve the problem and is proved to be convergent by using the contraction mapping theorem. For the case without knowing any information about the random system matrices, a normalized model-free value iteration algorithm is presented to learn the optimal control law by estimating the data for the next time. The collected data are normalized first in our model-free algorithm to reduce the error of the least squares method. It is proved that our algorithm can obtain an approximate optimal solution. Our algorithm is applicable for any distribution of the random system matrices and does not require an initial mean-square stabilizing control policy. Finally, an example illustrates that our algorithm converges to an approximate optimal control policy and that the normalization step can significantly reduce convergence errors.

Linear quadratic stochastic optimal control with state- and control-dependent noises: A deterministic data approach

This paper investigates an infinite-horizon linear quadratic stochastic optimal control (LQSOC) problem with state- and control-dependent noises, in which two system matrices are unknown. First, a deterministic system is introduced and a relationship among its system trajectory, its control and the matrices to be solved is formulated. Subsequently, based on the deterministic system, an adaptive dynamic programming (ADP) algorithm is established to tackle the LQSOC problem. Then, the convergence analysis is presented by proving the equivalence between the proposed algorithm and an existing policy iteration algorithm. Finally, the effectiveness of the obtained algorithm is verified by a perturbed turbocharged diesel engine optimal control design example.

Lagrangian dual method for solving stochastic linear quadratic optimal control problems with terminal state constraints

A stochastic linear quadratic (LQ) optimal control problem with a pointwise linear equality constraint on the terminal state is considered. A strong Lagrangian duality theorem is proved under a uniform convexity condition on the cost functional and a surjectivity condition on the linear constraint mapping. Based on the Lagrangian duality, two approaches are proposed to solve the constrained stochastic LQ problem. First, a theoretical method is given to construct the closed-form solution by the strong duality. Second, an iterative algorithm, called augmented Lagrangian method (ALM), is proposed. The strong convergence of the iterative sequence generated by ALM is proved. In addition, some sufficient conditions for the surjectivity of the linear constraint mapping are obtained.

Human-in-the-Loop Behavior Modeling via an Integral Concurrent Adaptive Inverse Reinforcement Learning.

One goal of artificial intelligence (AI) research is to teach machines how to learn from humans, such that they can perform a certain task in a natural human-like way. In this article, an online adaptive inverse reinforcement learning (IRL) approach to human behavior modeling is proposed to enhance machine intelligence for a class of linear human-in-the-loop (HiTL) systems using the state data only, where the human behavior is described by a linear quadratic optimal control model with an unknown weighting matrix for the quadratic cost function. First, an integral concurrent adaptive law is developed to learn the human feedback gain matrix online using the demonstrated state data only, which removes the persistent excitation (PE) conditions required by traditional adaptive estimation approaches and thus is more in line with real applications. Then, with the learned feedback gain matrix, the IRL problem is formulated as a linear matrix inequality (LMI) optimization problem, which can be efficiently solved to retrieve the weighting matrix of the human cost function. Finally, a simulation example is provided to illustrate the effectiveness of the proposed approach.

Linear quadratic optimal control problems for stochastic evolution equations in infinite horizon

Linear quadratic optimal control problems for stochastic evolution equations in infinite horizon

Value functional and optimal feedback control in linear-quadratic optimal control problem for fractional-order system

In this paper, a finite-horizon optimal control problem involving a dynamical system described by a linear Caputo fractional differential equation and a quadratic cost functional is considered. An explicit formula for the value functional is given, which includes a solution of a certain Fredholm integral equation. A step-by-step feedback control procedure for constructing $ \varepsilon $-optimal controls with any accuracy $ \varepsilon > 0 $ is proposed. The basis for obtaining these results is the study of a solution of the associated Hamilton–Jacobi–Bellman equation with so-called fractional coinvariant derivatives.

A Constraint Homotopy Active Set Solver for Linear-Quadratic Optimal Control

A Constraint Homotopy Active Set Solver for Linear-Quadratic Optimal Control