Nonlinear Stochastic Optimal Control Research Articles

Moving least-squares approximations for linearly-solvable stochastic optimal control problems

Nonlinear stochastic optimal control problems are fundamental in control theory. A general class of such problems can be reduced to computing the principal eigenfunction of a linear operator. Here, we describe a new method for finding this eigenfunction using a moving least-squares function approximation. We use efficient iterative solvers that do not require matrix factorization, thereby allowing us to handle large numbers of basis functions. The bases are evaluated at collocation states that change over iterations of the algorithm, so as to provide higher resolution at the regions of state space that are visited more often. The shape of the bases is automatically defined given the collocation states, in a way that avoids gaps in the coverage. Numerical results on test problems are provided.

Nonlinear stochastic optimal control strategy of hysteretic structures

Referring to the formulation of physical stochastic optimal control of structures and the scheme of optimal polynomial control, a nonlinear stochastic optimal control strategy is developed for a class of structural systems with hysteretic behaviors in the present paper. This control strategy provides an amenable approach to the classical stochastic optimal control strategies, bypasses the dilemma involved in It<TEX>$\hat{o}$</TEX>-type stochastic differential equations and is applicable to the dynamical systems driven by practical non-stationary and non-white random excitations, such as earthquake ground motions, strong winds and sea waves. The newly developed generalized optimal control policy is integrated in the nonlinear stochastic optimal control scheme so as to logically distribute the controllers and design their parameters associated with control gains. For illustrative purposes, the stochastic optimal controls of two base-excited multi-degree-of-freedom structural systems with hysteretic behavior in Clough bilinear model and Bouc-Wen differential model, respectively, are investigated. Numerical results reveal that a linear control with the 1st-order controller suffices even for the hysteretic structural systems when a control criterion in exceedance probability performance function for designing the weighting matrices is employed. This is practically meaningful due to the nonlinear controllers which may be associated with dynamical instabilities being saved. It is also noted that using the generalized optimal control policy, the maximum control effectiveness with the few number of control devices can be achieved, allowing for a desirable structural performance. It is remarked, meanwhile, that the response process and energy-dissipation behavior of the hysteretic structures are controlled to a certain extent.

Stochastic optimal time-delay control of quasi-integrable Hamiltonian systems

A nonlinear stochastic optimal time-delay control strategy for quasi-integrable Hamiltonian systems is proposed. First, a stochastic optimal control problem of quasi-integrable Hamiltonian system with time-delay in feedback control subjected to Gaussian white noise is formulated. Then, the time-delayed feedback control forces are approximated by the control forces without time-delay and the original problem is converted into a stochastic optimal control problem without time-delay. After that, the converted stochastic optimal control problem is solved by applying the stochastic averaging method and the stochastic dynamical programming principle. As an example, the stochastic time-delay optimal control of two coupled van der Pol oscillators under stochastic excitation is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy.

Necessary Conditions for Optimal Control of Stochastic Evolution Equations in Hilbert Spaces

We consider a nonlinear stochastic optimal control problem associated with a stochastic evolution equation. This equation is driven by a continuous martingale in a separable Hilbert space and an unbounded time-dependent linear operator.

Robustness of non-linear stochastic optimal control for quasi-Hamiltonian systems with parametric uncertainty

The robustness of non-linear stochastic optimal control for quasi-Hamiltonian systems with uncertain parameters is studied. Based on the independence of uncertain parameters and stochastic excitations, the non-linear stochastic optimal control for the nominal quasi-Hamiltonian system with average-value parameters is first obtained by using the stochastic averaging method and stochastic dynamical programming principle. Then, the means and standard deviations of root-mean-square responses, control effectiveness and control efficiency for the uncertain quasi-Hamiltonian system are calculated by using the stochastic averaging method and the probabilistic analysis. By introducing the sensitivity of the variation coefficients of controlled root-mean-square responses, control effectiveness and control efficiency to those of uncertain parameters, the robustness of the non-linear stochastic optimal control is evaluated. Two examples are given to illustrate the proposed control procedure and its robustness.

Feedback maximization of reliability of MDOF quasi integrable-Hamiltonian systems under combined harmonic and white noise excitations

We studied the feedback maximization of reliability of multi-degree-of-freedom (MDOF) quasi integrable-Hamiltonian systems under combined harmonic and white noise excitations. First, the partially averaged Ito equations are derived by using the stochastic averaging method for quasi integrable-Hamiltonian systems under combined harmonic and white noise excitations. Then, the dynamical programming equation and its boundary and final time conditions for the control problems of maximizing the reliability is established from the partially averaged equations by using the dynamical programming principle. The nonlinear stochastic optimal control for maximizing the reliability is determined from the dynamical programming equation and control constrains. The reliability function of optimally controlled systems is obtained by solving the final dynamical programming equation. Finally, the application of the proposed procedure and effectiveness of the control strategy are illustrated by using an example.

Nonlinear stochastic optimal control of Preisach hysteretic systems

The nonlinear stochastic optimal control of Preisach hysteretic systems is studied, and the control procedure is illustrated with an example of the single-degree-of-freedom Preisach system. The Preisach hysteretic system subjected to a stochastic excitation is first replaced by an equivalent non-hysteretic nonlinear stochastic system with displacement-amplitude-dependent damping and stiffness, by using the generalized harmonic balance technique. Then, the relationship between the displacement amplitude and total system energy is established, and the equivalent damping and stiffness coefficients are expressed as functions of the system energy. The averaged Itô stochastic differential equation for the system energy as one-dimensional controlled diffusion process, is derived by using the stochastic averaging method of energy envelope. For the semi-infinite time-interval ergodic control, the dynamical programming equation is obtained based on the stochastic dynamical programming principle, and is solved to yield the optimal control force. Finally, the Fokker–Planck–Kolmogorov equation associated with the averaged Itô equation is established, and the stationary probability density of the system energy is obtained, from which the variances of the controlled system response and the optimal control force are predicted and the control efficacy is evaluated. Numerical results show that the proposed control strategy for Preisach hysteretic systems is very effective and efficient.

Stochastic modeling of the dynamics of incident-induced lane traffic states for incident-responsive local ramp control

Incident-induced traffic congestion has been recognized as a critical issue to solve in the development of advanced freeway incident management systems. This paper investigates the applicability of a stochastic optimal control approach to real-time incident-responsive local ramp control on freeways. The architecture of the proposed ramp control system embeds two primary functions including (1) real-time estimation of incident-induced lane traffic states and (2) dynamic prediction of ramp-metering rates in response to the changes of incident impacts. To accomplish the above two goals, a discrete-time nonlinear stochastic optimal control model is proposed, followed by the development of a recursive prediction algorithm. Based on the simulation data, the numerical results of model tests indicate that the proposed method permits relieving incident impacts particularly under low-volume and medium-volume conditions, relative to high-volume lane-blocking conditions. Particularly, the incident-induced queue lengths can be improved by 50.1% and 67.9%, compared to the existing ramp control and control-free strategies, respectively.

Feedback minimization of first-passage failure of quasi integrable Hamiltonian systems

A nonlinear stochastic optimal control strategy for minimizing the first-passage failure of quasi integrable Hamiltonian systems (multi-degree-of-freedom integrable Hamiltonian systems subject to light dampings and weakly random excitations) is proposed. The equations of motion for a controlled quasi integrable Hamiltonian system are reduced to a set of averaged Ito stochastic differential equations by using the stochastic averaging method. Then, the dynamical programming equations and their associated boundary and final time conditions for the control problems of maximization of reliability and mean first-passage time are formulated. The optimal control law is derived from the dynamical programming equations and the control constraints. The final dynamical programming equations for these control problems are determined and their relationships to the backward Kolmogorov equation governing the conditional reliability function and the Pontryagin equation governing the mean first-passage time are separately established. The conditional reliability function and the mean first-passage time of the controlled system are obtained by solving the final dynamical programming equations or their equivalent Kolmogorov and Pontryagin equations. An example is presented to illustrate the application and effectiveness of the proposed control strategy.

Nonlinear Stochastic Dynamics and Control in Hamiltonian Formulation

AbstractThe significant advances in nonlinear stochastic dynamics and control in Hamiltonian formulation during the past decade are reviewed. The exact stationary solutions and equivalent nonlinear system method of Gaussian-white -noises excited and dissipated Hamiltonian systems, the stochastic averaging method for quasi Hamiltonian systems, the stochastic stability, stochastic bifurcation, first-passage time and nonlinear stochastic optimal control of quasi Hamiltonian systems are summarized. Possible extension and applications of the theory are pointed out. This review article cites 158 references.

Nonlinear stochastic optimal control of offshore platforms under wave loading

A nonlinear stochastic optimal (NSO) control strategy for wave-excited jacket-type offshore platforms is proposed. Wave force is determined according to linearized Morison equation. Spectral density functions of water particle velocity and acceleration are approximated by some rational forms, respectively. Wave force vector is then treated as output of a linear filter driven by white noise. A set of partially averaged Itô equations for controlled modal energies are derived by applying stochastic averaging method for quasi-integrable Hamiltonian systems. Optimal control law is then determined by using stochastic dynamical programming principle. To demonstrate the effectiveness and efficiency of the proposed control strategy, performances of uncontrolled, linear quadratic Gaussian (LQG)-controlled and NSO-controlled example platforms under different sea states are evaluated. Numerical results show that the NSO controller has better control effectiveness and efficiency than the LQG controller.

Bank management via stochastic optimal control

This paper examines a problem related to the optimal risk management of banks in a stochastic dynamic setting. In particular, we minimize market and capital adequacy risk that involves the safety of the securities held and the stability of sources of funds, respectively. In this regard, we suggest an optimal portfolio choice and rate of bank capital inflow that will keep the loan level as close as possible to an actuarially determined reference process. This set-up leads to a nonlinear stochastic optimal control problem whose solution may be determined by means of the dynamic programming algorithm. The above analysis is reliant on the construction of continuous-time stochastic models for bank behaviour upon which a spread method for loan capitalization is imposed.

Stochastic optimal control of first-passage failure for coupled Duffing–van der Pol system under Gaussian white noise excitations

Abstract In this paper, the stochastic averaging method of quasi-non-integrable-Hamiltonian systems is applied to Duffing–van der Pol system to obtain partially averaged Ito stochastic differential equations. On the basis of the stochastic dynamical programming principle and the partially averaged Ito equation, dynamical programming equations for the reliability function and the mean first-passage time of controlled system are established. Then a non-linear stochastic optimal control strategy for coupled Duffing–van der Pol system subject to Gaussian white noise excitation is taken for investigating feedback minimization of first-passage failure. By averaging the terms involving control forces and replacing control forces by the optimal ones, the fully averaged Ito equation is derived. Thus, the feedback minimization for first-passage failure of controlled system can be obtained by solving the final dynamical programming equations. Numerical results for first-passage reliability function and mean first-passage time of the controlled and uncontrolled systems are compared in illustrative figures to show effectiveness and efficiency of the proposed method.

Nonlinear stochastic optimal control of tall buildings under wind loading

A nonlinear stochastic optimal (NSO) control strategy for wind-excited tall buildings is proposed. The power spectral density (PSD) matrix of the fluctuating part of wind velocity vector is diagonalized in eigenvector space. Each element of the diagonalized PSD matrix is modeled as a set of second-order linear filters driven by white noise. A stochastic optimal control problem of partially observable system is formulated and converted into a stochastic optimal control problem of completely observable system based on the separation principle. The latter problem is solved by using the NSC strategy based on the stochastic averaging method for quasi-Hamiltonian systems and the dynamic programming principle. The response statistics of both uncontrolled and controlled structures is predicted analytically. A numerical example is given to show better control effectiveness and efficiency of the proposed NSO controller than those of linear quadratic Gaussian (LQG) controller.

Optimal feedback control of strongly non-linear systems excited by bounded noise

A strategy for non-linear stochastic optimal control of strongly non-linear systems subject to external and/or parametric excitations of bounded noise is proposed. A stochastic averaging procedure for strongly non-linear systems under external and/or parametric excitations of bounded noise is first developed. Then, the dynamical programming equation for non-linear stochastic optimal control of the system is derived from the averaged Itô equations by using the stochastic dynamical programming principle and solved to yield the optimal control law. The Fokker–Planck–Kolmogorov equation associated with the fully completed averaged Itô equations is solved to give the response of optimally controlled system. The application and effectiveness of the proposed control strategy are illustrated with the control of cable vibration in cable-stayed bridges and the feedback stabilization of the cable under parametric excitation of bounded noise.

Non-linear stochastic optimal control for coupled-structures system of multi-degree-of-freedom

Coupled structures under random excitation are modelled as a quasi-integrable Hamiltonian system of multi-degree-of-freedom and the reduced-order model in structural mode space is formulated. A non-linear stochastic optimal control method for the system is presented. The non-linear optimal control of adjacent tall building structures coupled with supplemental control devices and under random seismic excitation is performed by using the proposed method. First, applying the stochastic averaging method to the system yields It o ̂ stochastic differential equations for modal vibration energy processes, so that the system energy control is conducted generally instead of the system state control and the dimension of the control problem is reduced. Then applying the stochastic dynamical programming principle to the controlled diffusion processes yields a dynamical programming equation, taking into account random excitation spectra. An explicit polynomial solution to the equation is proposed to determine the non-linear optimal control forces. Furthermore, the response statistics of the controlled non-linear coupled structures under random seismic excitation are evaluated by using the stochastic averaging method, and are compared with those of the uncontrolled structures to determine the control efficacy. Numerical results illustrate the high control effectiveness and efficiency of the proposed non-linear stochastic optimal control method for coupled structures as a quasi-integrable Hamiltonian system.

Feedback minimization of first-passage failure of quasi non-integrable Hamiltonian systems

The non-linear stochastic optimal control of quasi non-integrable Hamiltonian systems for minimizing their first-passage failure is investigated. A controlled quasi non-integrable Hamiltonian system is reduced to an one-dimensional controlled diffusion process of averaged Hamiltonian by using the stochastic averaging method for quasi non-integrable Hamiltonian systems. The dynamical programming equations and their associated boundary and final time conditions for the problems of maximization of reliability and of maximization of mean first-passage time are formulated. The optimal control law is derived from the dynamical programming equations and the control constraints. The dynamical programming equations for maximum reliability problem and for maximum mean first-passage time problem are finalized and their relationships to the backward Kolmogorov equation for the reliability function and the Pontryagin equation for mean first-passage time, respectively, are pointed out. The boundary condition at zero Hamiltonian is discussed. Two examples are worked out to illustrate the application and effectiveness of the proposed procedure.

Nonlinear stochastic optimal control of partially observable linear structures

A strategy for nonlinear stochastic optimal control of partially observable linear structures is proposed and illustrated with linear building structures equipped with control devices and sensors under horizontal ground acceleration excitation. The control problem of a partially observable structure is first converted into that of a completely observable structure based on the separation principle. Then, a partially averaged control system of Itô equations is obtained from the completely observable structure by using the stochastic averaging method for quasi-Hamiltonian systems. Dynamical programming equations for finite and infinite time-interval controls are established based on the stochastic dynamical programming principle and solved to obtain the optimal control law and value function. Finally, the response of controlled structure is obtained from solving the Fokker–Planck–Kolmogorov equation associated with the fully averaged system of Itô equations. The numerical results for a five-story building structure model are obtained by using the proposed control strategy and compared with those by using linear quadratic Gaussian control strategy to show the effectiveness and efficiency of the proposed strategy.

A Monte-Carlo Approach to Nonlinear Stochastic Optimal Control with an Example Using the HMMS Data

Abstract This paper proposes a modified version of an algorithm for nonlinear stochastic control. The application of the algorithm is with the classic cost data from Holt, Muth, Modigliani and Simon (1955, 1960). New equations are reestimated using more recent econometrics techniques without requiring linear or quadratic forms, and the algorithm is applied for finding the optimal mix: workers, production. It is argued that the objective function must suffer two alterations: a correction factor to account for the difference between the stochastic solution and the deterministic solution, and a measure of skewness is included.

A General Stochastic Maximum Principle for Optimal Control Problems

The maximum principle for nonlinear stochastic optimal control problems in the general case is proved. The control domain need not be convex, and the diffusion coefficient can contain a control variable.