Multidimensional Control Problems Research Articles

Solving Singular Control from Optimal Switching

This report summarizes some of our recent work (Guo and Tomecek, SIAM J Control Optim 47(1):421–443, 2008; A class of singular control problems and smooth fit principle, 2008) on a new theoretical connection between singular control of finite variation and optimal switching problems. This correspondence not only provides a novel method for analyzing multi-dimensional singular control problems, but also builds links among singular controls, Dynkin games, and sequential optimal stopping problems.

Minimizing Sequences in Class-Qualified Deposit Problems

We study multidimensional control problems involving first-order partial differential equations. To ensure the existence of sufficiently regular multipliers (from the space \({C^{\ast}}\)) in the first-order necessary optimality conditions, some restrictions of the feasible domain have to be added. In particular, we investigate ‘class-qualified’ problems where the weak derivatives of \(x\) can be represented within a Baire function class. In the present paper, we prove conditions under which the original and the modified problems possess the same minimal values.

Economic dynamics of reservoir sedimentation management: Optimal control with singularly perturbed equations of motion

We analyze the economic dynamics of reservoir sedimentation management using the hydrosuction-dredging sediment-removal system. System dynamics depend on two interdependent hydraulic processes evolving at different rates. The accumulation of water impounded in the reservoir evolves on a ‘fast’ time scale, while the loss of water storage capacity to trapped sediments evolves on a ‘slow’ time scale. We formulate a multidimensional optimal control problem with singularly perturbed equations of motion to accommodate the disparate time scales. We apply singular perturbation methods to approximate (via polynomial series expansion) a ‘slow’ manifold reducing multi-dimensional solution space to the single-dimensional subspace confining long-term dynamics.

Piecewise Continuous Controls in Dieudonné-Rashevsky Type Problems

This paper considers multidimensional control problems governed by a first-order PDE system. It is known that, if the structure of the problem is linear-convex, then the so-called e-maximum principle, a set of necessary optimality conditions involving a perturbation parameter e > 0, holds. Assuming that the optimal controls are piecewise continuous, we are able to drop the perturbation parameter within the conditions, proving the Pontryagin maximum principle with piecewise regular multipliers (measures). The Lebesgue and Hahn decompositions of the multipliers lead to refined maximum conditions. Our proof is based on the Baire classification of the admissible controls.

AN OPTIMAL MARKOVIAN QUANTIZATION ALGORITHM FOR MULTI-DIMENSIONAL STOCHASTIC CONTROL PROBLEMS

We propose a probabilistic numerical method based on optimal quantization to solve some multi-dimensional stochastic control problems that arise, for example, in mathematical finance for portfolio optimization. We then consider some controlled diffusions with most components control free. The Euler scheme of the uncontrolled diffusion part is approximated by a discrete time process obtained by a nearest neighbor projection on some grids optimally fitted to its dynamics. The resulting process is also designed to preserve the Markov property with respect to the filtration of the Euler scheme. This Markovian quantization approach leads to an approximate control problem that can be solved numerically by the dynamic programming formula. This approach seems promising in higher dimension. A prioriLp-error bounds are stated and we show that the spatial discretization error term is minimal at some specific grids. A simple recursive algorithm is devised to compute these optimal grids by induction based on a Monte Carlo simulation. Some numerical illustrations are processed for solving a mean-variance hedging problem.

Pontryagin Principle for State-Constrained Control Problems Governed by a First-Order PDE System

This paper considers multidimensional control problems governed by a first-order PDE system and state constraints. After performing the standard Young measure relaxation, we are able to prove the Pontryagin principle by means of an ∈-maximum principle. Generalizing the common setting of one-dimensional control theory, we model piecewise-continuous weak derivatives as functions of the first Baire class and obtain regular measures as corresponding multipliers. In a number of corollaries, we derive necessary optimality conditions for local minimizers of the state-constrained problem as well as for global and local minimizers of the unconstrained problem.

Dynamic programming for multidimensional stochastic control problems

In this paper we study a general multidimensional diffusion-type stochastic control problem. Our model contains the usual regular control problem, singular control problem and impulse control problem as special cases. Using a unified treatment of dynamic programming, we show that the value function of the problem is a viscosity solution of certain Hamilton-Jacobi-Bellman (HJB) quasivariational inequality. The uniqueness of such a quasi-variational inequality is proved.

Optimal control of a divergent multi-echelon inventory system

Consider a divergent multi-echelon inventory system, such as a distribution system or a production system. At every facility in the system orders are placed (or production is initiated) periodically. The order arrives after a fixed lead time. At the end of each period linear costs are incurred penalty costs are incurred at the most downstream facilities for back-orders. The objective is to minimize the expected holding and penalty costs per period. We prove that under the balance assumption it is cost optimal to control every facility by an order-up-to-policy. The optimal replenishment policy, i.e. the order-up-to-level and the allocation functions at each facility, can be determined by system decomposition. This decomposition reduces complex multi-dimensional control problems to simple one-dimensional problems.

Parallel Algorithms for Stochastic Dynamic Programming with Continuous State and Control Variables

We compare two partitioning methods for solving a multi-dimensional optimal control problem in parallel using continuous state, continuous control, stochastic dynamic programming (SCP). The algorithm uses tensor products to interpolate the multi-dimensional, continuous variable state space. Unlike previous parallel SDP applications, an iterative optimization routine is used to find the optimal continuous control for each node point in the state space. This routine causes load balance and dependence problems not encountered in previous parallel SDP applications. Even with these additional computational complexities, superb efficiencies are achieved on a shared memory MIMD architecture. The two methods of solution differ in the following way: Method 1 uses the outer state loop index to send subsets of the state space (called parcels) to each concurrent process. The number of parcels sent as parallel tasks is the number of discretizations of one of the state variables. Method 2 allows the user to determine the number of parcels to be sent as parallel tasks. Method 2 splits the state space into a greater number of parcels, so that the processors are more likely to finish at the same time at a synchronization barrier. Computational results indicate that Method 2 has better load balance, with efficiencies up to 93.5%. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

Stability of the Barto-Sutton Network under Rational Control Orbits

The class of networks based on the Barto-Sutton architecture are known to be capable of solving complex, multi-dimensional control problems. In these problems, the objective of the task is the localization of a system within a contiguous region of its state space. In this work, asymptotic stability criteria are derived for the Adaptive Critical Element (ACE) of the network. Here, the weights of the network are viewed as the state of a linear time-variant state- space learning machine. For system trajectories which can be represented by simple rational polynomials, discrete-time techniques are used to analyze the stability of the learning machine. The advantages of this approach are that it both provides bounds for the learning parameters and characterizes the resultant learning behavior.

Mathematical Bioeconomics: The Optimal Management of Renewable Resources.

Introduction. 1. Elementary Dynamics of Exploited Populations. 1.1 The Logistic Growth Model. 1.2 Generalized Logistic Models: Depensation. 1.3 Summary and Critique. 2. Economic Models of Renewable-Resource Harvesting. 2.1 The Open-Access Fishery. 2.2 Economic Overfishing. 2.3 Biological Overfishing. 2.4 Optimal Fishery Management. 2.5 The Optimal Harvest Policy. 2.6 Examples Based on the Schaefer Model. 2.7 Linear Variational Problems. 2.8 The Possibility of Extinction. 2.9 Summary and Critique. 3. Capital-Theoretic Aspects of Resource Management. 3.1 Interest and Discount Rates. 3.2 Capital Theory and Renewable Resources. 3.3 Nonautonomous Models. 3.4 Applications to Policy Problems: Labor Mobility in the Fishery. 4. Optimal Control Theory. 4.1 One-Dimensional Control Problems. 4.2 A Nonlinear Fishery Model. 4.3 Economic Interpretation of the Maximum Principle. 4.4 Multidimensional Optimal Control Problem. 4.5 Optimal Investment in Renewable-Resource Harvesting. 5. Supply and Demand: Nonlinear Models. 5.1 The Elementary Theory of Supply and Demand. 5.2 Supply and Demand in Fisheries. 5.3 Nonlinear Cost Effects: Pulse Fishing. 5.4 Game-Theoretic Models. 5.5 Transboundary Fishery Resources: A Further Application of the Theory. 5.6 Summary and Critique. 6. Dynamical Systems. 6.1 Basic Theory. 6.2 Dynamical Systems in the Plane: Linear Theory. 6.3 Isoclines. 6.4 Nonlinear Plane-Autonomous Systems. 6.5 Limit Cycles. 6.6 Gause's Model of Interspecific Competition. 7. Discrete-Time and Metered Models. 7.1 A General Metered Stock-Recruitment Model. 7.2 The Beverton-Holt Stock-Recruitment Model. 7.3 Depensation Models. 7.4 Overcompensation. 7.5 A Simple Cohort Model. 7.6 The Production Function of a Fishery. 7.7 Optimal Harvest Policies. 7.8 The Discrete Maximum Principle. 7.9 Dynamic Programming. 8. The Theory of Resource Regulation. 8.1 A Behavioral Model. 8.2 Optimization Analysis. 8.3 Limited Entry. 8.4 Taxes and Allocated Transferable Quotas. 8.5 Total Catch Quotas. 8.6 Summary and Critique. 9. Growth and Aging. 9.1 Forestry Management: The Faustmann Model. 9.2 The Beverton-Holt Fisheries Model. 9.3 Dynamic Optimization in the Beverton-Holt Model. 9.4 The Case of Bounded F. 9.5 Multiple, Cohorts: Nonselective Gear. 9.6 Pulse Fishing. 9.7 Multiple Cohorts: Selective Gear. 9.8 Regulation. 9.9 Summary and Critique. 10. Multispecies Models. 10.1 Differential Productivity. 10.2 Harvesting Competing Populations. 10.3 Selective Harvesting. 10.4 A Diffusion Model: The Inshore-Offshore Fishery. 10.5 Summary and Critique. 11. Stochastic Resource Models. 11.1 Stochastic Dynamic Programming. 11.2 A Stochastic Forest Rotation Model. 11.3 Uncertainty and Learning. 11.4 Searching for Fish. 11.5 Summary and Critique. Supplementary Reading. References. Index.

Sufficiency Conditions for Weak Local Minima in Multidimensional Optimal Control Problems with Mixed Control-State Restrictions

In [13] a new sufficiency criterion for strong local minimality in multidimensional non-convex control problems with pure state constraint was developed. In this paper we use a similar method to obtain sufficient conditions for weak local minimality in multidimensional control problems with mixed state-control restrictions. The result is obtained by applying duality theory for control problems of Klötzler [11] as well as first and second order optimality conditions for optimization problems described by C^1 -functions having a locally Lipschitzian gradient mapping. The main theorem contains the result of Zeidan [17] for one-dimensional problems withoutstate restrictions.

Sufficient Conditions for Local Optimality in Multidimensional Control Problems with State Restrictions

New sufficient conditions for strong local minimality in multidimensional control pro-blems with state restrictions are presented. The results are obtained by applying duality theory and second order sufficient optimality conditions for optimization problems with functions having a locally Lipschitzian gradient mapping.

Erweiterung von mehrdimensionalen steuerungspromemen und dualität

Multidimensional control problems and corresponding dual problems are investigated. Results about strong duality between dual and generalized control problems are obtained and weak compactness of the set of generalized controls is proved

Towards sub-optimal stochastic control of partially observable stochastic systems

A class of multidimensional stochastic control problems with noisy data and bounded controls encountered in aerospace design is examined. The emphasis is on suboptimal design, the optimality being taken in quadratic mean sense. To that effect the problem is viewed as a stochastic version of the Lurie problem known from nonlinear control theory. The main result is a separation theorem (involving a nonlinear Kalman-like filter) suitable for Lurie-type approximations. The theorem allows for discontinuous characteristics. As a byproduct the existence of strong solutions to a class of non-Lipschitzian stochastic differential equations in dimensions is proven.

Multi-Dimensional Quality Control Problems and Quasi Variational Inequalities

Multi-Dimensional Quality Control Problems and Quasi Variational Inequalities

Multidimensional quality control problems and quasivariational inequalities

A machine can manufacture any one of n m-dimensional Brownian motions with drift λ j {\lambda _j} , P x λ j P_x^{{\lambda _j}} , defined on the space of all paths x ( t ) ∈ C ( [ 0 , ∞ ) ; R m ) x\left ( t \right )\, \in \,C\left ( {\left [ {0,\,\infty } \right );\,{R^m}} \right ) . It is given that the product is a random evolution dictated by a Markov process θ ( t ) \theta \left ( t \right ) with n states, and that the product is P x λ j P_x^{{\lambda _j}} when θ ( t ) = j , 1 ⩽ j ⩽ n \theta \left ( t \right )\, = \,j,\,1\, \leqslant \,j\, \leqslant \,n . One observes the σ \sigma -fields of x ( t ) x\left ( t \right ) , but not of θ ( t ) \theta \left ( t \right ) . With each product P x λ j P_x^{{\lambda _j}} there is associated a cost c j {c_j} . One inspects θ \theta at a sequence of times (each inspection entails a certain cost) and stops production when the state θ = n \theta \, = \,n is reached. The problem is to find an optimal sequence of inspections. This problem is reduced to solving a certain elliptic quasi variational inequality. The latter problem is actually solved in a rather general case.