Dual Control Problem Research Articles

Dual criterion stochastic optimal control problem for robustness improvement

The solution of the dual criterion linear quadratic stochastic optimal control problem is obtained by following a Wiener type of solution procedure. A stabilizing solution is guaranteed by parameterizing the controller using the Desoer fractional representation approach. The dual criterion includes sensitivity and complementary sensitivity weighting terms which provide a means of varying the robustness characteristics of the multivariable system.

Adaptive optimal estimation control strategies for systems of simultaneous equations

The choice of control strategies to improve estimation of the parameters in a model of a simultaneous equations system with time-varying parameters is considered. Open-loop feedback (OLF) sequential procedures for handling nonlinear restrictions on reduced form parameters implied by the structural form are suggested, and the combination of sequential estimation and design control strategies feature a marked improvement in the behavior of estimates over the nonsequential [open-loop (OL)] formulation. The maximum accuracy control problem considered in this paper can also be treated as an initial phase of a forecasting and/or stochastic control problem. This will avoid solution to a more difficult problem such as the dual control problem.

Some existence and regularity results for dual linear control problems

Some existence and regularity results for dual linear control problems

Simple nonlinear dual control problems

AbstractIn this paper we consider a simple, nonlinear optimal control problem with sufficient convexity to enable us to formulate its dual problem. Both primal and dual problems will include constraints on both the states and controls. The constraints in one problem may cause the “optimal” dual states to be discontinuous. However, we will look at conditions under which the presence of constraints does not force discontinuities and the optimal states and costates are absolutely continuous.

Adaptive dual control of discrete-time distributed-parameter stochastic systems

Abstract The problem of adaptive dual control of discrete-time distributed-parameter stochastic systems is examined. It is shown that there exists an important difference between feedback and closed-loop policies of control for this type of system as for the lumped parameter case. This difference is based on the adaptivity feature of the control. Namely, when the control policy affects both the state and its uncertainty (dual effect) it possesses the so-called feature of active adaptivity and can only be a characteristic of a closed-loop policy, whereas a feedback policy can only be passively adaptive. These results can be used to develop a control algorithm for non-linear problems for which the realization of optimal control laws involves control strategies with both learning and control features.

Aspects of Optimal Estimation Control Strategies in Econometrics

In this paper we investigate the class of control strategies that are optimal for estimation in the context of an econometric model where attention is limited to & admissible sample points which have been chosen so that the relevant region is adequately covered. A new sequential procedure for handling nonlinear restrictions on reduced form coefficients implied by the structural form is suggested. This procedure is shown to resolve the conceptual difficulties inherent in alternative formulations. The maximum accuracy problem considered in this paper can also be treated as an initial phase of a stochastic control problem. This will avoid solution to a difficult dual control problem.

Learning Control of Finite Markov Chains

Two learning algorithms are presented for the Markovian decision problem in which the transition probabilities are unknown. The algorithms select decisions each time on the basis of the estimates of the unknown probabilities. While one makes a probabilistic selection and the other a deterministic one, they are both devised in view of the dual control problem. It is shown that the probabilistic algorithm converges to an optimal policy. On the other hand, it is shown that the deterministic one only achieves an ε-optimal frequency of selecting an optimal policy. But the selecting procedure of the latter is much simpler than that of the former.

A comment on a dual control problem

It is shown that a certain dual control problem can be reduced in two different ways to two standard completely observed stochastic control problems.

Estimation and Optimal Control of an Uncertain Production Process

AbstractThree alternative decision rules for estimation and control of an uncertain production process are examined. One is a certainty equivalence strategy whereby an optimal rate based on current estimates of production parameters is used throughout the planning horizon. A second is updated certainty equivalence, a passively adaptive strategy. Third is an actively adaptive control formulation. Monte‐Carlo simulation results show that the passively adaptive strategy outperforms the others. For the cases simulated, the actively adaptive approximation to the information state for a dual control problem is not good enough to establish superiority of this sophisticated strategy for all problems.

A Dual Approach to a Linear Smoothing Problem

In this paper, the continuous-time linear Gaussian smoothing problem is investigated by utilizing a dual approach. It is shown that the smoothing problem is a dual of an optimal regulator with a jump condition on the trajectory. By solving the dual optimal control problem, basic results established earlier on this smoothing problem are derived in a simpler way. Results are also obtained for the discrete-time linear smoothing Gaussian problem.

Toward a computationally efficient optimal solution to the LQG discrete-time dual control problem

A computationally attractive optimal solution to the discrete-time linear-quadratic-Gaussian (LQG) dual control problem in the absence of plant noise is presented. Convex vector parametric uncertainties are allowed and no a priori information is assumed save that the uncertain vector is an element of a known compact subset of Rp. It is shown that game theoretic techniques are useful provided an incremental quadratic loss function is chosen. The suboptimal solution is easily implemented since it is an average of a finite number of LQG controllers weighted by easily generated likelihood ratios. Furthermore, the structure lends itself easily to approximate solutions to the time-varying parameter case. However, further research is required to simplify the current cumbersome design procedure.

On geometric duality for state-constrained control problems

For convex optimal control problems without explicit pure state constraints, the structure of dual problems is now well known. However, when these constraints are present and active, the theory of duality is not highly developed. The major difficulty is that the dual variables are not absolutely continuous functions as a result of singularities when the state trajectory hits a state constraint. In this paper we recognize this difficulty by formulating the dual probram in the space of measurable functions. A strong duality theorem is derived. This pairs a primal, state constrained convex optimal control problem with a dual convex control problem that is unconstrained with respect to state constraints. In this sense, the dual problem is computationally more attractive than the primal.

On the differentiability in dual optimal control problems

A suitable dualization is used for construction of unconstrained dual programs to a certain rather general class of optimal control problems. Using the effective tool of convex analysis, conditions are specified, under which costs of those dual programs are differentiable. The developed theory is illustrated on examples representing simultaneously main areas of applicability of the presented approach.

Primal (quantity), dual (price) linear optimal control problems and the weak duality theorem

In the continuation primal and dual optimal control problems are formulated. The disticntion between state and control variables, state equations and control variable constraints are discarded. A general treatment is achieved by the introduction of a matrix F. The derivation of the dual from the primal follows approximately the approach by J.B. Rosen. The corresponding weak duality theorem is deduced. It is then indicated how typical optimal control formulations may be obtained by the specialisation of the matrices and vectors and their substitution.

Sequential strategies in dual control problems

Sequential strategies in dual control problems

A separation theorem for stochastic control problems with point-process observations

The exact solution is derived for a stochastic optimal control problem involving a linear stochastic plant, quadratic costs, and nonlinear, nongaussian observations. The observations are in the form of a point process in which each point has both a temporal and a spatial coordinate. The state of the stochastic plant influences the intensity of the observed time-space point process. The solution to this dual control problem can be realized with a separated estimator-controller in which the estimator is nonlinear, mean-square optimal, and finite dimensional, and the controller is the certainty equivalent linear controller. Motivation for the stochastic optimal control problem studied here is given in terms of position sensing and tracking for quantum-limited optical communication problems.

A simple dual control problem with an analytical solution

A stochastic control problem for which the optimal dual control law can be calculated analytically is given. The system is a four state Markov chain with transition probabilities that depend on the control variable. The performance of the optimal dual control law and of some suboptimal control laws are calculated and compared.

Convex programming in a space adjoint to a banach space and convex optimal control problems with phase constraints

THE CONDITIONS are found under which a problem of convex programming in a space adjoint to a Banach space is linked by duality relations with a problem of linear programming in the initial space. The results are applied to an optimal control problem with phase constraints. If the control of this latter problem is sought in the class of functions L∞[0, T], measurable and bounded allmost everywhere in [0, T], then L∞[0, T] is the Banach space, adjoint to L 1[0, T]. The dual optimal control problem proves to be linear, with controls in L 1[0, T]. The conditions are found under which the Pontryagin maximum principle can be written in a form containing no measures [1]. These conditions are easy to check and are satisfied for many problems of mathematical economics.

Stochastic adaptive control methods: a survey

There is a vast literature about adaptive (self-organizing, self-optimizing) control systems. The aim of this paper is to point out some of the main ideas. Since there exists good survey papers about several of the classes of adaptive controllers the main part of this paper will concentrate on recent developments. The paper will cover stochastic adaptive controllers. The problem of dual control is given particular attention.

A synthesis of learning dual control for single-input/single-output systems

Abstract The stochastic dual control problem is considered for linear discrete-time single-input/ single-output systems with unknown parameters. The optimal strategy for simultaneous identification and control is obtained analytically for the expected value of a specific quadratic criterion in which a vector of n past outputs and n— I past inputs is used as the state. The combined problem thus reduces to a linear adaptive problem, providing complete state information. The matrix Riccati difference equation and the linear feedback law are modified, fitting for a class of random parameter systems. Then it is found that when the identification process converges, the control strategy for systems with constant but unknown parameters converges to that for known parameter systems as time tends to infinity.