Completion Of Squares Research Articles

Decentralized Linear Quadratic Systems With Major and Minor Agents and Non-Gaussian Noise

A decentralized linear quadratic system with a major agent and a collection of minor agents is considered. The major agent affects the minor agents, but not vice versa. The state of the major agent is observed by all agents. In addition, the minor agents have a noisy observation of their local state. The noise processes is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">not</i> assumed to be Gaussian. The structures of the optimal strategy and the best linear strategy are characterized. It is shown that major agent's optimal control action is a linear function of the major agent's MMSE (minimum mean squared error) estimate of the system state while the minor agent's optimal control action is a linear function of the major agent's MMSE estimate of the system state and a “correction term” which depends on the difference of the minor agent's MMSE estimate of its local state and the major agent's MMSE estimate of the minor agent's local state. Since the noise is non-Gaussian, the minor agent's MMSE estimate is a non-linear function of its observation. It is shown that replacing the minor agent's MMSE estimate by its LLMS (linear least mean square) estimate gives the best linear control strategy. The results are proved using a direct method based on conditional independence, common-information-based splitting of state and control actions, and simplifying the per-step cost based on conditional independence, orthogonality principle, and completion of squares.

Mean–variance portfolio selection under no-shorting rules: A BSDE approach

This paper revisits the mean–variance portfolio selection problem in continuous-time within the framework of short-selling of stocks is prohibited via backward stochastic differential equation approach. To relax the strong condition in Li et al. (Li et al. 2002), the above issue is formulated as a stochastic recursive optimal linear–quadratic control problem. Due to no-shorting rules (namely, the portfolio taking non-negative values), the well-known “completion of squares” no longer applies directly. To overcome this difficulty, we study the corresponding Hamilton–Jacobi–Bellman (HJB, for short) equation inherently and derive the two groups of Riccati equations. On one hand, the value function constructed via Riccati equations is shown to be a viscosity solution of the HJB equation mentioned before; On the other hand, by means of these Riccati equations and backward semigroup, we are able to get explicitly the efficient frontier and efficient investment strategies for the recursive utility mean–variance portfolio optimization problem.

Indefinite risk-sensitive control

We consider a risk-sensitive control problem with indefinite cost matrices. All solutions to such an optimal control problem are obtained in an explicit closed-form by the completion of squares and the change of measure methods. Both the finite and infinite horizon cases are considered. As an application, we find all solutions to an optimal investment problem in a market with a stochastic interest rate.

Optimal regulators for a class of nonlinear stochastic systems

We consider a class of nonlinear stochastic systems with square-root nonlinearities appearing in the diffusion terms. The optimal control problems with indefinite quadratic criteria in both finite and infinite horizon are formulated and solved in an explicit closed form. It turns out that all optimal controls are of an affine state-feedback form, despite the fact that the system is nonlinear. We use the method of completion of squares and new types of Riccati differential and algebraic equations to find the solutions. An application to the problem of optimal investment in a market with a stochastic interest rate is given.

Linear–quadratic mean field stochastic zero-sum differential games

In this paper, we consider the two-player linear–quadratic mean field stochastic differential game (LQ-MF-SZSDG), where the expected values of state and players’ control variables are included in the corresponding mean field stochastic system and the objective functional. We characterize the explicit condition under which the feedback saddle-point equilibrium for the LQ-MF-SZSDG exists. The results are obtained by using the “completion of squares” method and the “ordered interchangeability” property of multiple saddle points. We show that the corresponding saddle-point equilibrium is linear in state, which is characterized by the two coupled Riccati differential equations (CRDEs). We also discuss the solvability conditions of the CRDEs. Numerical results are presented to verify the theoretical results of the paper.

A kind of optimal investment problem under inflation and uncertain time horizon

In this paper, we study a kind of optimal investment problem under inflation and uncertain time horizon. It can be generally formulated into a stochastic optimal control problem. In particular for the constant relative risk aversion utility, we employ the method of completion of squares to give an explicit form of optimal portfolio and maximum utility by the solution of a stochastic Riccati equation, whose wellposedness is obtained and also of significance in its own right. The most distinguishing result of our work is that the randomness of exit time actually affects not only the optimal portfolio but also the maximum utility in the case of stochastic market parameters. Moreover, we present several numerical examples to show the application of theoretical results and further discuss the influence of inflation and random time horizon from the economic viewpoint.

Optimal Local and Remote Controllers With Unreliable Uplink Channels: An Elementary Proof

Recently, a model of a decentralized control system with local and remote controllers connected over unreliable channels was presented in [1] . The model has a nonclassical information structure that is not partially nested. Nonetheless, it is shown in [1] that the optimal control strategies are linear functions of the state estimate (which is a nonlinear function of the observations). Their proof is based on a fairly sophisticated dynamic programming argument. In this article, we present an alternative and elementary proof of the result which uses common information-based conditional independence and completion of squares.

Optimal insurance control for insurers with jump-diffusion risk processes

AbstractIn this paper, we model the surplus process as a compound Poisson process perturbed by diffusion and allow the insurer to ask its customers for input to minimize the distance from some prescribed target path and the total discounted cost on a fixed interval. The problem is reduced to a version of a linear quadratic regulator under jump-diffusion processes. It is treated using three methods: dynamic programming, completion of square and the stochastic maximum principle. The analytic solutions to the optimal control and the corresponding optimal value function are obtained.

The variational method in quantum mechanics: an elementary introduction

Variational methods in quantum mechanics are customarily presented as invaluable techniques to find approximate estimates of ground state energies. In the present paper a short catalogue of different celebrated potential distributions (both 1D and 3D), for which an exact and complete (energy and wavefunction) ground state determination can be achieved in an elementary way, is illustrated. No previous knowledge of calculus of variations is required. Rather, in all presented cases the exact energy functional minimization is achieved by using only a couple of simple mathematical tricks: ‘completion of square’ and integration by parts. This makes our approach particularly suitable for undergraduates. Moreover, the key role played by particle localization is emphasized through the entire analysis. This gentle introduction to the variational method could also be potentially attractive for more expert students as a possible elementary route toward a rather advanced topic on quantum mechanics: the factorization method. Such an unexpected connection is outlined in the final part of the paper.

Optimal decentralized control of two agent linear system with partial output feedback: certainty equivalence and optimality of linear strategies

Abstract We consider the optimal decentralized control of two agent linear system in which the agent 2’s state is affected by the state and control actions of agent 1 but not vice versa. The state of agent 1 is perfectly observed by both agents while the state of agent 2 is observed with noise by agent 2. Thus, the information structure is partially nested. However, we do not assume that the process and observation noises are Gaussian. Therefore, it is not known a priori whether linear strategies are globally optimal. Without using dynamic programming, we show that the optimal strategy is linear and certainty equivalent. Our proof is based on three steps: static reduction via state splitting, orthogonal projection, and completion of squares. We believe that our solution methodology is of independent interest for decentralized control of linear systems.

Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients

We are concerned with the linear-quadratic optimal stochastic control problem where all the coefficients of the control system and the running weighting matrices in the cost functional are allowed to be predictable (but essentially bounded) processes and the terminal state-weighting matrix in the cost functional is allowed to be random. Under suitable conditions, we prove that the value field $\mathbb{V}(t,x,\omega), (t,x,\omega)\in [0,T]\times R^n\times \Omega$, is quadratic in $x$ and has the following form: $\mathbb{V}(t,x)=\langle \mathbb{K}_tx, x\rangle$, where $\mathbb{K}$ is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that $\mathbb{K}$ is a continuous semimartingale of the form $\mathbb{K}_t=\mathbb{K}_0-\int_0^t \, dk_s+\int_0^t\sum_{i=1}^d L_s^i\, dW_s^i, \ t\in [0,T]$, with $k$ being a continuous process of bounded variation and the stochastic integral $\int_0^\cdot\sum_{i=1}^d L^i_s\, dW^i_s$ being a bounded mean oscillation martingale, and that $(\mathbb{K}, L)$ with $L:=(L^1, \ldots, L^d)$ is a solution to the associated backward stochastic Riccati equation (BSRE), whose generator is highly nonlinear in the unknown pair of processes. The uniqueness is also proved via a localized completion of squares in a self-contained manner for a general BSRE. The existence and uniqueness of an adapted solution to a general BSRE was initially proposed by the French mathematician J. M. Bismut [SIAM J. Control Optim., 14 (1976), pp. 419--444; Lecture Notes in Math. 649, C. Dellacherie, P. A. Meyer, and M. Weil, eds., Springer-Verlag, Berlin, 1978, pp. 180--264] and subsequently listed by S. Peng [Control of Distributed Parameter and Stochastic Systems (Hangzhou, 1998), S. Chen et al., eds., Kluwer, Boston, 1999, pp. 265--273] as an open problem for backward stochastic differential equations. It had remained open until a general solution by the author [SIAM J. Control Optim., 42 (2003), pp. 53--75] via the stochastic maximum principle with a viewpoint of stochastic flow for the associated stochastic Hamiltonian system. The present paper is its companion and gives the second but more comprehensive (seemingly much simpler, but appealing to the advanced tool of Doob--Meyer's decomposition theorem and the general theory of stochastic processes, in addition to the DPP) adapted solution to a general BSRE via the DPP. Further extensions to the jump-diffusion control system and to the general nonlinear control system are possible.

Some linear-quadratic stochastic differential games for equations in Hilbert spaces with fractional Brownian motions

A noncooperative, two person, zero sum, stochastic differential game is formulated and solved that is described by a linear stochastic equation in a Hilbert space with a fractional Brownian motion and a quadratic payoff functional for the two players. The stochastic equation can model stochastic partial differential equations not only with distributed strategies and noise but also with control strategies and noise restricted to the boundary of the domain. The optimal strategies for the two players are given explicitly. The verification method is a generalization of completion of squares and provides the optimal strategies directly without solving partial differential equations or backward stochastic differential equations. Some examples of games described by stochastic partial differential equations are given.

Risk-sensitive control for a class of nonlinear systems with multiplicative noise

In this paper, we consider the problem of optimal control for a class of nonlinear stochastic systems with multiplicative noise. The nonlinearity consists of quadratic terms in the state and control variables. The optimality criteria are of a risk-sensitive and generalised risk-sensitive type. The optimal control is found in an explicit closed-form by the completion of squares and the change of measure methods. As applications, we outline two special cases of our results. We show that a subset of the class of models which we consider leads to a generalised quadratic–affine term structure model (QATSM) for interest rates. We also demonstrate how our results lead to generalisation of exponential utility as a criterion in optimal investment.

A direct method for solving stochastic control problems

In this paper a method motivated by completion of squares is used to describe explicit optimal controls for some stochastic control problems that include the linear-quadratic control problem for systems with a general noise process, the linear exponential quadratic Gaussian control problem for systems with Brownian motion, and the control of Brownian motion in the two sphere and the real hyperbolic plane with both finite and infinite time horizons. 1. Introduction. While the study of stochastic control problems can be traced from the evolution of the calculus of variations and deterministic control and from various stochastic control problems in statistics, the first major problem solution for the formally defined area of optimal stochastic control was the linear-quadratic Gaus- sian (LQG) problem (e.g. (2), (3)) that is also called the stochastic regulator problem (e.g. (13)). The area of stochastic control has developed significantly in breadth and depth from the solution of the LQG problem approximately fifty years ago. Two general methods have developed for solving stochastic control problems. They are the Hamilton-Jacobi-Belllman (HJB) equation and the stochastic maximum (or min- imum) principle. The HJB equation can be considered as a natural generalization of the Hamilton-Jacobi equation of classical mechanics and the stochastic maximum principle can be considered as a natural generalization of the maximum principle of deterministic control (e.g. (18)). For a stochastic control problem the Hamilton-Jacobi-Bellman equation is (typ- ically) a nonlinear second order partial differential equation. The control system is assumed to generate a continuous time Markov process for any control from the family of admissible controls. Clearly the questions of existence and uniqueness of solutions for HJB equations are difficult in general. Therefore the notion of solutions is of- ten relaxed to viscosity solutions (14). However an optimal control that arises from the solution of an HJB equation may not be in the family of admissible controls. Thus significant difficulties often arise with this method though it provides sufficient conditions for optimality. The stochastic maximum principle method is another important approach which yields necessary conditions for optimality. With some suitable convexity conditions � Dedicated to Professor Hanfu Chen on the occasion of his 75th birthday. Research supported by NSF grants DMS 0808138 and DMS 1108884, AFOSR grants FA9550-09-12-1-0384 and FA9550-12-

ℋ2 and ℋ∞ robust output feedback control for continuous time polytopic systems

Robust dynamic output feedback design is an open problem, computationally speaking, since its determination asks for the solution of nonlinear matrix inequalities, namely bilinear ones. This is particularly the case for polytopic uncertainty. Here, new sufficient conditions for ℋ2 and ℋ∞ robust output feedback control synthesis are proposed by the use of bounds and scaling for completion of squares. The usefulness of the provided conditions stands in the fact that its solution can be performed using the Frank–Wolfe algorithm which runs in only one shot. The ℋ2 robust control design of an inverted pendulum with uncertain friction coefficients and ℋ∞ reliable control systems design illustrate the theory.

Another look at finite horizon H∞ control problems for systems with input delays

We discuss a finite horizon H ∞ control problem for time-varying systems with input delays. Clarifying a relationship between two H ∞ control problems in input delay case and in measurement delay case, we derive a solution in input delay case based on the known result for the H ∞ control problem in measurement delay case, and show that the solution has the same predictor–observer structure as the solution in measurement delay case has. Using this structural information on the solution, we also present an elementary proof of the solution to the finite horizon H ∞ control problem for systems with input delays, which is based only on completion of squares.

Dynamic Mean-Variance Portfolio Selection with No-Shorting Constraints

This paper is concerned with mean-variance portfolio selection problems in continuous-time under the constraint that short-selling of stocks is prohibited. The problem is formulated as a stochastic optimal linear-quadratic (LQ) control problem. However, this LQ problem is not a conventional one in that the control (portfolio) is constrained to take nonnegative values due to the no-shorting restriction, and thereby the usual Riccati equation approach (involving a completion of squares) does not apply directly. In addition, the corresponding Hamilton--Jacobi--Bellman (HJB) equation inherently has no smooth solution. To tackle these difficulties, a continuous function is constructed via two Riccati equations, and then it is shown that this function is a viscosity solution to the HJB equation. Solving these Riccati equations enables one to explicitly obtain the efficient frontier and efficient investment strategies for the original mean-variance problem. An example illustrating these results is also presented.

Another look at finite horizon H∞ control problems for systems with input delays

We discuss a finite horizon H∞ control problem for systems with input delays. Clarifying a relationship between two H∞ control problems in input delay case and in measurement delay case. we derive a solution in input delay case based on the known result for the H∞ control problem in measurement delay case. and show that the solution has the same predictor-observer structure as the solution in measurement delay case has. Using this structural infonnation on the solution. We also present a direct proof of the solution to the finite horizon Hoo control problem for systelns with input delays. which is based only on completion of squares.

Inverse optimal design of input-to-state stabilizing nonlinear controllers

We show that input-to-state stabilizability (as defined by Sontag,1989, 1995) is both necessary and sufficient for the solvability of a Hamilton-Jacobi-Isaacs equation associated with a meaningful differential game problem similar to, but more general than, the "nonlinear H/sub /spl infin//" problem. The significance of the result stems from the fact that constructive solutions to the input-to-state stabilization problem are available (presented in the paper) and that, as shown here, inverse optimal controllers possess margins on input-to-state stability against a certain class of input unmodeled dynamics. Rather than completion of squares, the main tools in our analysis are Legendre-Fenchel transformations and the general form of Young's inequality.

MixedH2/H∞control via state feedback for nonlinear systems

This paper studies a mixed H 2/H ∞ control problem with internal stability by means of memoryless state feedback, for smooth nonlinear control systems. The problem consists of combining the performance requirements of quadratic optimal controllers with the robustness properties of H∞ controllers. It is shown that under appropriate assumptions, the problem is solvable if a pair of cross-coupled Hamilton-Jacobi-Isaacs equations allows smooth solutions. Our approach to the mixed H 2/H ∞ nonlinear control problem is drawn from the concepts of non-zero sum differential game, LaSalle's Invariance Principle and completion of squares. Analogous results are also developed for discrete-time nonlinear systems, by using the Taylor expansion argument. Finally, all the results obtained in affine systems are generalized to non-affine nonlinear systems.