Infinite Horizon Optimal Control Research Articles

Infinite Horizon Optimal LQG Control of Interconnected Systems With Application to Multiarea Power Systems

Infinite Horizon Optimal LQG Control of Interconnected Systems With Application to Multiarea Power Systems

Locally bounded variations epigraph property of the value function to infinite horizon optimal control problems under state constraints

Locally bounded variations epigraph property of the value function to infinite horizon optimal control problems under state constraints

Optimal control of Boolean control networks with state-triggered impulses

In the real world, abrupt changes caused by external environment and system interaction can exhibit a pattern resembling an “impulse” response. A novel model, known as the hybrid-index model, has recently been introduced for Boolean control networks (BCNs) to characterize such impulsive responses. The optimal control problem is a fundamental issue in control theory, and many control problems can be classified as special cases of optimal control problems. Nevertheless, the optimal control problem for the hybrid-index model remains unresolved. This study delves into the optimal control problem of impulsive Boolean control networks (IBCNs) under the hybrid-index model framework. Forward completeness serves as a crucial condition to avert Zeno behavior in the hybrid-index model of IBCNs. In contrast to prior studies on the hybrid-index model of IBCNs, the study shows that the optimal control problem of IBCNs can be addressed even in cases where the system is not forward complete. Leveraging the theory of semi-tensor product, both finite-horizon and infinite-horizon optimal control problems of IBCNs are tackled using the quotient mapping method and the recursive method for optimal control of Boolean control networks. The study yields the optimal cost and optimal control law through state feedback. To illustrate the efficacy of the proposed approach, a mathematical example and a biological example known as the λ switch are presented.

An analytical adaptive optimal control approach without solving HJB equation for nonlinear systems with input constraints

AbstractThis paper presents an analytical method to solve the optimal control problem for affine nonlinear systems with unknown drift dynamics. A new non‐quadratic cost function over an infinite horizon is presented that considers input constraints and includes the cost of the feed‐forward component of the control law. The mean value theorem for vector‐valued functions has been used to derive an integral form of this theorem. Based on this theorem, a rigorous proof is provided demonstrating that the cost function can be converted into another form. In the presence of input constraints, this converted form enables extracting the optimal control solution without solving the HJB equation. Additionally, unknown nonlinearity effects in drift dynamics are compensated in the control input. This is accomplished by estimating the unknown drift dynamics via an adaptive neural network (NN) approach. It is proven that the states and weights of NN are uniformly ultimately bounded based on a Lyapunov technique. The necessary and sufficient conditions are provided that ensure the optimality of the infinite horizon optimal control problem with a discount factor. As a result, it is demonstrated that the proposed approach satisfies the optimality criteria. To evaluate the effectiveness of the proposed approach, simulation examples are provided.

Reinforcement learning for optimal control of linear impulsive systems with periodic impulses

This paper focuses on the finite- and infinite-horizon optimal control problems of linear impulsive systems with periodic impulses under the quadratic performance index. Necessary and sufficient conditions for the optimal impulsive system are derived in terms of hybrid Riccati equations by utilizing the variational method and the collocation method combined with a time-varying Lyapunov function. Different from the existing model-based impulsive control schemes, three reinforcement learning (RL)-based algorithms are proposed to solve the optimal impulsive controller and hybrid controller for the impulsive system without the exact knowledge of the system dynamics. The asymptotical stability of the impulsive control system and the convergence of the RL-based algorithms are proved rigorously. Finally, a numerical simulation illustrates the effectiveness of the proposed control methods.

Policy gradient methods for discrete time linear quadratic regulator with random parameters

This paper studies an infinite horizon optimal control problem for discrete-time linear system and quadratic criteria, both with random parameters which are independent and identically distributed with respect to time. In this general setting, we apply the policy gradient method, a reinforcement learning technique, to search for the optimal control without requiring knowledge of statistical information of the parameters. We investigate the sub-Gaussianity of the state process and establish global linear convergence guarantee for this approach based on assumptions that are weaker and easier to verify compared to existing results. Numerical experiments are presented to illustrate our result.

Dissipativity in Infinite Horizon Optimal Control and Dynamic Programming

In this paper we extend dynamic programming techniques to the study of discrete-time infinite horizon optimal control problems on compact control invariant sets with state-independent best asymptotic average cost. To this end we analyse the interplay of dissipativity and optimal control, and propose novel recursive approaches for the solution of so called shifted Bellman Equations.

Compactification method in linear programming approach to infinite-horizon optimal control problems with a noncompact state constraint

Compactification method in linear programming approach to infinite-horizon optimal control problems with a noncompact state constraint

Direct integral pseudospectral and integral spectral methods for solving a class of infinite horizon optimal output feedback control problems using rational and exponential Gegenbauer polynomials

This study is concerned with the numerical solution of a class of infinite-horizon linear regulation problems with state equality constraints and output feedback control. We propose two numerical methods to convert the optimal control problem into nonlinear programming problems (NLPs) using collocations in a semi-infinite domain based on rational Gegenbauer (RG) and exponential Gegenbauer (EG) basis functions. We introduce new properties of these basis functions and derive their quadratures and associated truncation errors. A rigorous stability analysis of the RG and EG interpolations is also presented. The effects of various parameters on the accuracy and efficiency of the proposed methods are investigated. The performance of the developed integral spectral method is demonstrated using two benchmark test problems related to a simple model of a divert control system and the lateral dynamics of an F-16 aircraft. Comparisons of the results of the current study with available numerical solutions show that the developed numerical scheme is efficient and exhibits faster convergence rates and higher accuracy.

Conditional Cost Function and Necessary Optimality Conditions for Infinite Horizon Optimal Control Problems

Conditional Cost Function and Necessary Optimality Conditions for Infinite Horizon Optimal Control Problems

CONDITIONAL COST FUNCTION AND NECESSARY OPTIMALITY CONDITIONS FOR INFINITE HORIZON OPTIMAL CONTROL PROBLEMS

Infinite horizon optimal control problem with general endpoint constraints is reduced to a family of standard problems on finite time intervals containing the value of the conditional cost of the phase vector as a terminal term. New version of the Pontryagin maximum principle containing an explicit characterization of the adjoint variable is obtained for the problem with a general asymptotic endpoint constraint. In the case of the problem with free final state this approach leads to a normal form version of the maximum principle formulated completely in the terms of the conditional cost function.

Accessibility and stabilization by infinite horizon optimal control with negative discounting

Accessibility and stabilization by infinite horizon optimal control with negative discounting

Stability and Performance Analysis of NMPC: Detectable Stage Costs and General Terminal Costs

We provide a stability and performance analysis for nonlinear model predictive control (NMPC) schemes subject to input constraints. Given an exponential stabilizability and detectability condition w.r.t. the employed state cost, we provide a sufficiently long prediction horizon to ensure asymptotic stability and a desired performance bound w.r.t. the infinite-horizon optimal controller. Compared to existing results, the provided analysis is applicable to positive semi-definite (detectable) cost functions, provides tight bounds using a linear programming analysis, and allows for a seamless integration of general positive-definite terminal cost functions in the analysis. The practical applicability of the derived theoretical results are demonstrated with numerical examples.

Mathematical Model of Human Capital Dynamics

A mathematical description of household economic behavior is studied. On the one hand, households are consumers that seek to maximize the discounted utility function in an imperfect market of savings and consumer loans. On the other hand, households are workers in the labor market; they receive a wage and seek to enhance their skills to receive a higher wage. An increase in the level of worker’s skill is achieved via investment in human capital. In this paper, a mathematical model of the worker’s behavior in the labor market is represented in the form of an infinite-horizon optimal control problem. A solution existence theorem is proved, and necessary optimality conditions are obtained in the form of Pontryagin’s maximum principle. The model is identified using Russian statistical data for various social layers.

Necessary Conditions for the Optimality and Sustainability of Solutions in Infinite-Horizon Optimal Control Problems

The paper deals with an infinite-horizon optimal control problem with general asymptotic endpoint constraints. The fulfillment of constraints of this type can be viewed as the minimal necessary condition for the sustainability of solutions. A new version of the Pontryagin maximum principle with an explicitly specified adjoint variable is developed. The proof of the main results is based on the fact that the restriction of the optimal process to any finite time interval is a solution to the corresponding finite-horizon problem containing the conditional cost of the phase vector as a terminal term.

Infinite horizon fuzzy optimal control: optimality does not imply asymptotic stability

In this paper, novel stability result for discrete-time infinite horizon optimal control using fuzzy objective functions is presented. For this class of control, the fuzzy goals and the fuzzy constraints introduced in the fuzzy objective function handle the constraints placed on both the state and the control vectors. We analyze the asymptotic stability of the equilibrium for the infinite horizon fuzzy optimal control law using the minimum aggregation operator. We show that the infinite horizon control with the minimum aggregation operator does not guarantee the asymptotic stability of the equilibrium in general. This is done by deriving an analytical solution of the control law for a simple linear system using a fuzzy dynamic programming approach. An example that shows the novel asymptotic stability result of the equilibrium for discrete-time infinite horizon optimal control with fuzzy objective function problem is given.

On optimal control for linear distributed time-delay systems

This article presents the solution to the infinite horizon optimal control problem for linear distributed time-delay systems. It is based on the use of the Cauchy solution for distributed time-delay systems. In contrast to previous results for systems with concentrated delays, the form of the functional and its properties are formally justified. An important property is demonstrated: the Bellman functional admits upper and lower bounds; thus, it is proved to be positive definite. Additionally, some experimental results of a temperature process are presented.

Infinite Horizon Optimal Control Problems with Discount Factor on the State, Part II: Analysis of the Control Problem

This is the continuation of our work on infinite horizon optimal control problems with a discount factor on the state variable and nonlinear partial differential equations as constraints. Existence of a solution is proven, and first as well as second order optimality conditions are derived. They are used to analyze the approximation of the infinite horizon problem by finite horizon problems.

Infinite Horizon Optimal Control for a General Class of Semilinear Parabolic Equations

A class of infinite horizon optimal control problems subject to semilinear parabolic equations is investigated. First and second order optimality conditions are obtained, in the presence of constraints on the controls, which can be either pointwise in space-time, or pointwise in time and L^2 in space. These results rely on a new L^infty estimate for nonlinear parabolic equations in an essential manner.

Infinite Horizon Optimal Control Problems with Discount Factor on the State, Part I: Analysis of the Controlled State Equation

This is the first part of an investigation of infinite horizon optimal control problems subject to semilinear parabolic equations. A discount factor on the state variable is introduced in the cost. This allows for the treatment of infinite horizon problems without stabilizability assumptions. The nonlinearities can be of polynomial type, thus covering reaction-diffusion equations which are important for applications. The control-to-state mapping and its regularity are analyzed in detail. This involves the relation between the type of the nonlinearity and the discount factor.