Optimal Nonlinear Feedback Control Research Articles

Nonlinear Optimal Control for Stochastic Dynamical Systems

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system’s asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers.

Finite-Time Stabilization and Optimal Feedback Control for Nonlinear Discrete-Time Systems

Finite-time stability involves dynamical systems whose trajectories converge to an equilibrium state in finite time. Sufficient conditions for finite-time stability have recently been developed in the literature for discrete-time dynamical systems. In this article, we build on these results to develop a framework for addressing the problem of optimal nonlinear analysis and feedback control for finite-time stability and finite-time stabilization for nonlinear discrete-time controlled dynamical systems. Finite-time stability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that satisfies a difference inequality involving fractional powers and a minimum operator. This Lyapunov function can clearly be seen to be the solution to a difference equation that corresponds to a steady-state form of the Bellman equation, and hence, guaranteeing both finite-time stability and optimality. Finally, a numerical example is presented to demonstrate the efficacy of the proposed finite-time discrete stabilization framework.

Theoretical Study of the Unconstrained and Constrained Nonlinear Optimal Discrete Time State Feedback Control

Abstract Majority of the optimal control techniques can only be applied successfully if the model of the controlled process is known and it is linear. If the system model is nonlinear, then this nonlinear model can be approximated with different simple, linear models. However, these models are valid only in the neighbourhood of the operating points. The success of the control algorithms is highly dependent on the used linearization methods. The aim of the paper is to compare different optimal control algorithms and linearization methods. The presented optimal control algorithms have been also tested in constrained and unconstrained versions.

Nonlinear Optimal Feedback Control of the Two-Level Open Non-Markovian Stochastic Quantum System

In this paper, a nonlinear optimal feedback control based on the nonlinear state estimator for a two-level open non-Markovian stochastic quantum system is proposed. The proposed nonlinear state estimator is designed by using the state-dependent differential Riccati equation and constructed to optimally estimate the state. The estimated state is updated by the output data of continuous weak measurement of the controlled quantum system and used to design the nonlinear state feedback controller to track the output of the reference model. The output state of reference model is chosen as the desired performance. The numerical simulation results verify the achievability of the proposed state feedback control strategy, and the capability to steer the state of system from any arbitrary initial state to the final reference target state with high state transfer success rate.

Policy iteration for Hamilton–Jacobi–Bellman equations with control constraints

Policy iteration is a widely used technique to solve the Hamilton Jacobi Bellman (HJB) equation, which arises from nonlinear optimal feedback control theory. Its convergence analysis has attracted much attention in the unconstrained case. Here we analyze the case with control constraints both for the HJB equations which arise in deterministic and in stochastic control cases. The linear equations in each iteration step are solved by an implicit upwind scheme. Numerical examples are conducted to solve the HJB equation with control constraints and comparisons are shown with the unconstrained cases.

A Reduced-order Model for Real-time NMPC of Ethanol Steam Reformers

One of the approaches to green energy of interest in recent years is through the use of hydrogen technologies in which the waste product of combustion is water rather than carbon dioxide, nitrous oxides, and other pollutants generated by the burning of hydrocarbon fuels. Ethanol steam reformers is one of the primary technologies with the potential to generate hydrogen efficiently and reliably. This article investigates the design of nonlinear optimal feedback control systems for ethanol steam reformers based on mechanistic models described by a system of nonlinear partial differential-algebraic equations, aka singular/descriptor systems. The mechanistic models are too complicated to simulate in real-time using the standard numerical discretization methods, much less to directly incorporate such models as nonlinear constraints into mechanistic model-based nonlinear model predictive control formulations. This article employs a rather sophisticated multivariate form of the method of characteristics to reformulate the nonlinear constraints into a much simpler form amenable to real-time control implementations.

SDRE-Based Integral Sliding Mode Control for Wind Energy Conversion Systems

This paper proposes a novel integral sliding mode control (ISMC) scheme based on numerically solving a state-dependent Ricatti equation (SDRE), nonlinear feedback control for wind energy conversion systems (WECSs) with permanent magnet synchronous generators (PMSGs). Unlike the conventional ISMC, the proposed control system is designed with nonlinear near optimal feedback control part to take into account nonlinearities of the WECSs. The Taylor series are used to approximate the solutions of SDRE. More specifically, the nonlinear optimal feedback control has been obtained by solving continuous algebraic Ricatti and Lyapunov equations. Sliding variables are designed such that reaching phase is eliminated and stability is guaranteed. The proposed control method equipped with high-order observer can guarantee more superior results than linear techniques such as linear quadratic regulator (LQR), conventional ISMC, and first-order sliding-mode control (SMC) method. Increasing the number of terms of the Taylor's series of the proposed control law provides better approximation, therefore the performance is improved. However, this increases the computational burden. The effectiveness of the control method is validated via simulations in MATLAB/Simulink under nominal parameters and model uncertainties.

Nonlinear Optimal Feedback Control and Stability Analysis of Solar Photovoltaic Systems

The efficiency of solar photovoltaic (PV) systems is directly affected by the convergence and steady-state responses of the implemented control method. In this paper, considering the nonlinearity appearing in the model of the solar PV system, we employ a nonlinear optimal feedback control scheme to deal with the oscillations around the maximum power point (MPP) of the system, induced by the chattering phenomenon in the control. Taking into account the improved transient response and flexibility, brought by including the cross-weighting terms in the cost functional, we develop an optimal control framework with a nonquadratic cost for addressing the MPP tracking (MPPT) problem of the solar PV system. Exploiting the fact that a Lyapunov function candidate can be considered as the steady-state solution of the Hamilton-Jacobi-Bellman (HJB) equation, we obtained the optimal feedback controller via minimizing the resultant Hamiltonian. The stability analysis of the closed-loop system is done for the obtained control law with a guaranteed performance measure. Moreover, to enhance the practicality of the obtained control law, we present two procedures to implement the obtained control scheme under nonuniform insolation and as a model-free approach, separately. To demonstrate the merits of the proposed framework, the obtained optimal feedback control, together with the partial shading condition and model-free approach, is simulated under various weather conditions. The optimal approach illustrates an improved performance in terms of the convergence rate and the amplitude of oscillations around the MPP, compared to existing results in the literature.

Implementation and Evaluation of Non-linear Optimal Feedback Control for Ship’s Automatic Berthing by Recurrent Neural Network

In this paper, we present an automatic ship’s berthing system by non-linear optimal feedback controller. In the proposed method, the recurrent neural network is used for non-linear optimal feedback controller for the realistic operational conditions such as different berthing distances and different disturbances. To obtain the feasible non-linear controller, the recurrent neural network is trained using pre-computed non-linear optimal solutions for various conditions. In order to evaluate the performance of the proposed system, extensive computer simulations and actual sea tests are carried out using small training ship Shioji-Maru under various conditions. As a result, we can see that the proposed non-linear optimal feedback controller by recurrent neural network is useful for automatic berthing system for the ship.

Solution of stochastic optimal control problems and financial applications

In this paper, the stochastic optimal control problems, which frequently occur in economic and finance are investigated. First, using Bellman’s dynamic programming method the stochastic optimal control problems are converted to Hamilton-Jacobi-Bellman (HJB) equation. Then, obtained HJB equation is solved through the method of separation of variables by guessing a solution via its terminal condition. Also, the non-linear optimal feedback control law is constructed. Finally, the solution procedure is illustrated for solving some examples that two of them are financial models. In fact, to highlight the applications of stochastic optimal control problems in financial mathematics, some financial models are presented.

Partial-State Stabilization and Optimal Feedback Control for Stochastic Dynamical Systems

In this paper, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control for partial stability and partial-state stabilization of stochastic dynamical systems. Partial asymptotic stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that is positive definite and decrescent with respect to part of the system state which can clearly be seen to be the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation, and hence, guaranteeing both partial stability in probability and optimality. The overall framework provides the foundation for extending optimal linear-quadratic stochastic controller synthesis to nonlinear-nonquadratic optimal partial-state stochastic stabilization. Connections to optimal linear and nonlinear regulation for linear and nonlinear time-varying stochastic systems with quadratic and nonlinear-nonquadratic cost functionals are also provided. Finally, we also develop optimal feedback controllers for affine stochastic nonlinear systems using an inverse optimality framework tailored to the partial-state stochastic stabilization problem and use this result to address polynomial and multilinear forms in the performance criterion.

Nonlinear–nonquadratic optimal and inverse optimal control for stochastic dynamical systems

SummaryIn this paper, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control for nonlinear stochastic dynamical systems. Specifically, we provide a simplified and tutorial framework for stochastic optimal control and focus on connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory. In particular, we show that asymptotic stability in probability of the closed‐loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady‐state form of the stochastic Hamilton–Jacobi–Bellman equation and, hence, guaranteeing both stochastic stability and optimality. In addition, we develop optimal feedback controllers for affine nonlinear systems using an inverse optimality framework tailored to the stochastic stabilization problem. These results are then used to provide extensions of the nonlinear feedback controllers obtained in the literature that minimize general polynomial and multilinear performance criteria. Copyright © 2017 John Wiley & Sons, Ltd.

Stochastic finite-time partial stability, partial-state stabilization, and finite-time optimal feedback control

In many practical applications, stability with respect to part of the system’s states is often necessary with finite-time convergence to the equilibrium state of interest. Finite-time partial stability involves dynamical systems whose part of the trajectory converges to an equilibrium state in finite time. In this paper, we address finite-time partial stability in probability and uniform finite-time partial stability in probability for nonlinear stochastic dynamical systems. Specifically, we provide Lyapunov conditions involving a Lyapunov function that is positive definite and decrescent with respect to part of the system state and satisfies a differential inequality involving fractional powers for guaranteeing finite-time partial stability in probability. In addition, we show that finite-time partial stability in probability leads to uniqueness of solutions in forward time and we establish necessary and sufficient conditions for almost sure continuity of the settling-time operator of the nonlinear stochastic dynamical system. Finally, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control design for finite-time partial stochastic stability and finite-time, partial-state stochastic stabilization. Finite-time partial stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that is positive definite and decrescent with respect to part of the system state and can clearly be seen to be the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation guaranteeing both finite-time, partial-state stability and optimality. The overall framework provides the foundation for extending stochastic optimal linear–quadratic controller synthesis to nonlinear–nonquadratic optimal finite-time, partial-state stochastic stabilization.

An IPEM for optimal control of uncertain beam-moving mass systems with saturation nonlinearity

This paper addresses the vibration control of single-span beams subjected to a moving mass by coupling the saturated nonlinear control and an improved point estimation method (IPEM). An optimal nonlinear feedback control law, for a kind of uncertain linear system with actuator nonlinearities, is derived using the combination of Pontryagin's maximum principles and the improved point estimation method. The stability of the feedback system is guaranteed using a Lyapunov function. In order to obtain the instantaneously probabilistic information of output responses, a novel moment approach is presented by combining the improved point estimation method, the maximum entropy methodology and the probability density evolution theory. In addition to the consideration of stochastic system parameters, the external loadings are considered as a nonstationary random excitation and a moving sprung mass, respectively. The proposed strategy is then used to perform vibration suppression analysis and parametric sensitivity analysis of the given beam. From numerical simulation results, it is deduced that the improved point estimation method is a priority approach to the optimal saturated nonlinear control of stochastic beam systems. This observation has widespread applications and prospects in vehicle–bridge interaction and missile–gun systems.

Sparse Approximation–Based Collocation Scheme for Nonlinear Optimal Feedback Control Design

This paper discusses the development of a computationally efficient approach to generate optimal feedback control laws for infinite time problems by solving the corresponding Hamilton–Jacobi–Bellman (HJB) equation. The solution process consists of iteratively solving the linear generalized HJB (GHJB) equation starting with an admissible stable controller. The collocation methods are exploited to solve the GHJB equation in the specified domain of interest. Recently developed nonproduct quadrature method known as Conjugate Unscented Transformation is used to manage the curse of dimensionality associated with the growth of collocation points with increase in the state dimension. Furthermore, recent advances in sparse approximation are leveraged to automatically generate the appropriate polynomial basis function set for the collocation-based approximation from an overcomplete dictionary of basis functions. It is demonstrated that the solution process uses the basis function selection process to automatically identify a form for the feedback control law, which is frequently unknown. Several numerical examples demonstrate the efficacy of the proposed approach in accurately generating feedback control policies for nonlinear systems.

Finite-Time Stabilization and Optimal Feedback Control

Finite-time stability involves dynamical systems whose trajectories converge to an equilibrium state in finite time. Since finite-time convergence implies nonuniqueness of system solutions in reverse time, such systems possess non-Lipschitzian dynamics. Sufficient conditions for finite-time stability have been developed in the literature using continuous Lyapunov functions. In this technical note, we develop a framework for addressing the problem of optimal nonlinear analysis and feedback control for finite-time stability and finite-time stabilization. Finite-time stability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that satisfies a differential inequality involving fractional powers. This Lyapunov function can clearly be seen to be the solution to a partial differential equation that corresponds to a steady-state form of the Hamilton-Jacobi-Bellman equation, and hence, guaranteeing both finite-time stability and optimality.

Partial‐state stabilization and optimal feedback control

SummaryIn this paper, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control for partial stability and partial‐state stabilization. Partial asymptotic stability of the closed‐loop nonlinear system is guaranteed by means of a Lyapunov function that is positive definite and decrescent with respect to part of the system state, which can clearly be seen to be the solution to the steady‐state form of the Hamilton–Jacobi–Bellman equation and hence guaranteeing both partial stability and optimality. The overall framework provides the foundation for extending optimal linear‐quadratic controller synthesis to nonlinear nonquadratic optimal partial‐state stabilization. Connections to optimal linear and nonlinear regulation for linear and nonlinear time‐varying systems with quadratic and nonlinear nonquadratic cost functionals are also provided. Finally, we also develop optimal feedback controllers for affine nonlinear systems using an inverse optimality framework tailored to the partial‐state stabilization problem and use this result to address polynomial and multilinear forms in the performance criterion. Copyright © 2015 John Wiley & Sons, Ltd.

Finite-time partial stability and stabilization, and optimal feedback control

In this paper, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control design for finite-time partial stability and finite-time, partial-state stabilization. Finite-time partial stability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that is positive definite and decrescent with respect to part of the system state, satisfies a differential inequality involving fractional powers, and can clearly be seen to be the solution to the steady-state form of the Hamilton-Jacobi-Bellman equation guaranteeing both partial stability and optimality. The overall framework provides the foundation for extending optimal linear-quadratic controller synthesis to nonlinear-nonquadratic optimal finite-time, partial-state stabilization. In addition, we specialize our results to address the problem of optimal finite-time control for nonlinear time-varying systems. Finally, we develop optimal feedback controllers for affine nonlinear systems using an inverse optimality framework tailored to the finite-time, partial-state stabilization problem and use this result to address finite-time, partial-state stabilizing sublinear controllers that minimize a derived performance criterion involving subquadratic terms.

Optimal nonlinear feedback control of spacecraft rendezvous with finite low thrust between libration orbits

This paper presents the nonlinear closed-loop feedback control strategy for the spacecraft rendezvous problem with finite low thrust between libration orbits in the Sun–Earth system. The model of spacecraft rendezvous takes the perturbations in initial states, the actuator saturation limits, the measurement errors, and the external disturbance forces into consideration from an engineering point of view. The proposed nonlinear closed-loop feedback control strategy is not analytically explicit; rather, it is implemented by a rapid re-computation of the open-loop optimal control at each update instant. To guarantee the computational efficiency, a novel numerical algorithm for solving the open-loop optimal control is given. With the aid of the quasilinearization method, the open-loop optimal control problem is replaced successfully by a series of sparse symmetrical linear equations coupled with linear complementary problem, and the computational efficiency can be significantly increased. The numerical simulations of spacecraft rendezvous problems in the paper well demonstrate the robustness, high precision, and dominant real-time merits of the proposed closed-loop feedback control strategy.

High order optimal feedback control of space trajectories with bounded control

Optimal feedback control is classically based on linear approximations, whose accuracy drops off rapidly in highly nonlinear dynamics. Several nonlinear optimal feedback control strategies have appeared in recent years. Among them, differential algebraic techniques have been used to tackle nonlinearities by expanding the solution of the optimal control problem about a reference trajectory and reducing the computation of optimal feedback control laws to the evaluation of high order polynomials. However, the resulting high order method could not handle control saturation constraints, which remain a critical facet of nonlinear optimal feedback control. This work introduces the management of saturating actuators in the differential algebraic method. More specifically, the constraints are included in the optimal control problem formulation and differential algebra is used to expand the associated optimal bang–bang solution with respect to the initial and terminal conditions. Optimal feedback control laws for thrust direction and switching times are again computed by evaluating the resulting polynomials. Illustrative applications are presented in the frame of the optimal low-thrust transfer to asteroid 1996 FG3.