Multidimensional Control Problems Research Articles

On generalized KT-pseudoinvex control problems involving multiple integral functionals

In this paper, we introduce (strongly) b-KT-pseudoinvex multidimensional control problems. In this regard, we consider an extended condition of invexity and we prove that (strongly) b-KT-pseudoinvex multidimensional control problems are described such that all Kuhn–Tucker points are optimal solutions. As well, in order to illustrate our main characterization result and its effectiveness, we propose an application.

KT‐pseudoinvex multidimensional control problem

SummaryIn this paper, we define a Kuhn‐Tucker (KT)–pseudoinvex multidimensional control problem. More exactly, we introduce a new condition on the functions, which are involved in a multidimensional control problem, and we prove that a KT‐pseudoinvex multidimensional control problem is characterized such that a KT point is an optimal solution. Thus, we generalize optimality results in known mathematical programming problems. These theoretical results are illustrated with an application.

Bias-Corrected Least-Squares Monte Carlo for Utility Based Optimal Stochastic Control Problems

The Least-Squares Monte Carlo (LSMC) method has gained popularity in recent years due to its ability to handle multi-dimensional stochastic control problems, including problems with state variables affected by control. However, when applied to the stochastic control problems in the multi-period expected utility models, the regression fit tends to contain errors which accumulate over time and typically blow up the numerical solution. In this paper we propose to transform the value function of the problems to improve the regression fit, and then using either the smearing estimate or smearing estimate with controlled heteroskedasticity to avoid the re-transformation bias in the estimates of the conditional expectations calculated in the LSMC algorithm. We also present and utilise recent improvements in the LSMC algorithms such as control randomisation with policy iteration to avoid accumulation of regression errors over time. Presented numerical examples demonstrate that transformation method leads to an accurate solution. In addition, in the forward simulation stage of the control randomisation algorithm, we propose a re-sampling of the state and control variables in their full domain at each time t and then simulating corresponding state variable at t+1, to improve the exploration of the state space that also appears to be critical to obtain a stable and accurate solution for the expected utility models.

A Combination of Variational and Penalty Methods for Solving a Class of Fractional Optimal Control Problems

This paper develops an approximate method, based on the combination of epsilon penalty and variational methods, for solving a class of multidimensional fractional optimal control problems. The fractional derivative is in the Caputo sense. In the presented method, utilizing the epsilon method, the given optimal control problem transforms into an unconstrained optimization problem; then, the equivalent variational equality is derived for the given unconstrained problem. The variational equality is approximately solved by applying a spectral method.

Models and algorithms for skip-free Markov decision processes on trees

We introduce a class of models for multidimensional control problems that we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in the negative direction. Starting with the finite average cost case, we show that the algorithm combines the advantages of both value iteration and policy iteration—it is guaranteed to converge to an optimal policy and optimal value function after a finite number of iterations but the computational effort required for each iteration step is comparable with that for value iteration. We show that the algorithm can also be used to solve discounted cost models and continuous-time models, and that a suitably modified algorithm can be used to solve communicating models.

An approximation scheme for stochastic controls in continuous time

We propose a simple time-discretization scheme for multi-dimensional stochastic optimal control problems in continuous time. It is based on a probabilistic representation for the convolution of the value function by a probability density function. We show the convergence results under mild conditions on coefficients of the problems by Barles–Souganidis viscosity solution method. Resulting numerical methods allow us to use uncontrolled Markov processes to estimate the conditional expectations in the dynamic programming procedure. Moreover, it can be implemented without the interpolation of the value function or the adjustment of the diffusion matrix.

An approximate method for numerically solving multi-dimensional delay fractional optimal control problems by Bernstein polynomials

In this paper, we present a method for solving a class of fractional optimal control problems with time delay. The method is based upon the Bernstein polynomial basis. The main reason of using this technique is the efficiency and simple application of Bernstein polynomials in dealing with dynamical systems, which contains constant delay. In this work, we use the fractional derivative in Caputo sense and explain our method for fractional derivative of order \(0 < \alpha \le 1\). Some numerical examples are given to illustrate the effectiveness of our method.

A Multidimensional Stochastic Singular Control Problem Via Dynkin Game and Dirichlet Form

The traditional difficulty with stochastic singular control is characterizing the regularities of the value function and the optimal control policy. In this paper, a multidimensional singular control problem is considered. We found the optimal value function and the optimal control policy of this problem via a Dynkin game, whose solution is given by the saddle point of the cost function. The existence and uniqueness of the solution to this Dynkin game are proved through an associated variational inequality problem involving Dirichlet form. As a consequence, the properties of the value function of this Dynkin game imply the smoothness of the value function of the stochastic singular control problem. In this way, we are able to show the existence of a classical solution to this multidimensional singular control problem, which was traditionally solved in the sense of viscosity solutions, and this enables the application of the verification theorem to prove optimality.

Solving multi-dimensional fractional optimal control problems with inequality constraint by Bernstein polynomials operational matrices

In this paper, we present a method for solving multi-dimensional fractional optimal control problems. Firstly, we derive the Bernstein polynomials operational matrix for the fractional derivative in the Caputo sense, which has not been done before. The main characteristic behind the approach using this technique is that it reduces the problems to those of solving a system of algebraic equations, thus greatly simplifying the problem. The results obtained are in good agreement with the existing ones in the open literature and it is shown that the solutions converge as the number of approximating terms increases, and the solutions approach to classical solutions as the order of the fractional derivatives approach 1.

A direct method for the solution of an optimal control problem arising from image registration

In the present paper, the the elastic/hyperelastic image registration problem is treated as a multidimensional control problem of Dieudonn&#233-Rashevsky type. For its numerical solution, we describe a direct method based on discretization methods and large-scale optimization techniques. Selected numerical results will be presented and discussed. The quality of the results obtained with the optimal control method competes well with those generated from a standard variational method.

ε‐Optimal Value and Approximate Multidimensional Dual Dynamic Programming

AbstractThis paper develops an approximate dual dynamic programming for an ε− optimal multidimensional control problem governed by first order hyperbolic equations. The problem considered is of the Dieudonne‐Rashevsky type and contains as a particular case class‐qualified deposit and Capital theory problems. It is proven that each Lipschitz continuous function satisfying the dual Hamilton‐Jacobi inequality can be used to define an ε‐optimal value.

A discretization method for the numerical solution of Dieudonné–Rashevsky type problems with application to edge detection within noisy image data

SUMMARYThe present paper is concerned with the numerical solution of multidimensional control problems of Dieudonné–Rashevsky type by discretization methods and large‐scale optimization techniques. We prove first a convergence theorem wherein the difference of the minimal value and the objective values along a minimizing sequence is estimated by the mesh size of the underlying triangulations. Then we apply the proposed method to the problem of edge detection within raw image data. Instead of using an Ambrosio–Tortorelli type energy functional, we reformulate the problem as a multidimensional control problem. The edge detector can be built immediately from the control variables. The quality of our numerical results competes well with those obtained by applying variational techniques. Copyright © 2011 John Wiley &amp; Sons, Ltd.

Quasiconvex relaxation of multidimensional control problems with integrandsf(t,ξ,v)

We prove a general relaxation theorem for multidimensional control problems of Dieudonne-Rashevsky type with nonconvex integrands f (t , ξ , v ) in presence of a convex control restriction. The relaxed problem, wherein the integrand f has been replaced by its lower semicontinuous quasiconvex envelope with respect to the gradient variable, possesses the same finite minimal value as the original problem, and admits a global minimizer. As an application, we provide existence theorems for the image registration problem with convex and polyconvex regularization terms.

A numerical scheme for two-dimensional optimal control problems with memory effect

A new formulation for multi-dimensional fractional optimal control problems is presented in this article. The fractional derivatives which are coming from the formulation of the problem are defined in the Riemann–Liouville sense. Some terminal conditions are imposed on the state and control variables whose dimensions need not be the same. A numerical scheme is described by using the Grünwald–Letnikov definition to approximate the Riemann–Liouville Fractional Derivatives. The set of fractional differential equations, which are obtained after the discretization of the time domain, are solved within the Grünwald–Letnikov approximation to obtain the state and the control variable numerically. A two-dimensional fractional optimal control problem is studied as an example to demonstrate the performance of the scheme.

Jensen's inequality for the lower semicontinuous quasiconvex envelope and relaxation of multidimensional control problems

Assume that K ⊂ R n m is a convex body with o ∈ int ( K ) and f : R n m → R ∪ { + ∞ } is a function with f | K ∈ C 0 ( K , R ) and f | ( R n m ∖ K ) ≡ + ∞ . We show that its lower semicontinuous quasiconvex envelope f ( q c ) ( w ) = sup { g ( w ) | g : R n m → R ∪ { + ∞ } quasiconvex and lower semicontinuous , g ( v ) ⩽ f ( v ) ∀ v ∈ R n m } obeys the Jensen's integral inequality f ( q c ) ( w ) = f ( q c ) ( ( ∫ K v 11 d ν ( v ) ⋯ ∫ K v 1 m d ν ( v ) ⋮ ⋮ ∫ K v n 1 d ν ( v ) ⋯ ∫ K v n m d ν ( v ) ) ) ⩽ ∫ K f ( q c ) ( ( v 11 ⋯ v 1 m ⋮ ⋮ v n 1 ⋯ v n m ) ) d ν ( v ) ∀ ν ∈ S ( q c ) ( w ) for every w ∈ K where S ( q c ) ( w ) is a subset of probability measures. This result is then applied to multidimensional control problems of Dieudonné–Rashevsky type: Relaxation by replacement of the integrand by its lower semicontinuous envelope and relaxation by introduction of generalized controls lead to problems with identical minimal values.

Sufficient optimality conditions for multidimensional optimal control problems with mixed constraints governed by an elliptic PDE

In this article, the idea of a dual dynamic programming is applied to the optimal control problems with multiple integrals governed by a semi-linear elliptic PDE and mixed state-control constraints. The main result called a verification theorem provides the new sufficient conditions for optimality in terms of a solution to the dual equation of a multidimensional dynamic programming. The optimality conditions are also obtained by using the concept of an optimal dual feedback control. Besides seeking the exact minimizers of problems considered some kind of an approximation is given and the sufficient conditions for an approximated optimal pair are derived.

Detection of Intensity and Motion Edges within Optical Flow via Multidimensional Control

In this paper, we propose a new optimization approach for the simultaneous computation of optical flow and edge detection therein. Instead of using an Ambrosio-Tortorelli type energy functional, we reformulate the optical flow problem as a multidimensional control problem. The optimal control problem is solved by discretization methods and large-scale optimization techniques. The edge detector can be immediately built from the control variables. We provide three series of numerical examples. The first shows that the mere presence of a gradient restriction has a regularizing effect, while the second demonstrates how to balance the regularizing effects of a term within the objective and the control restriction. The third series of numerical results is concerned with the direct evaluation of a TV-regularization term by introduction of control variables with sign restrictions.

A novel cautious controller to uncertain models arising in stochastic control

In this paper we present the Riccati solution of linear quadratic control problems with input and state dependent noise. The proposed solution will be referred to as the cautious Riccati solution. It is suitable for deterministic and stochastic systems characterised by functional uncertainty. Uncertainty of the system equations is quantified using a state and control dependent noise model. The derived optimal control law is shown to be of cautious type controllers. Since the problem considered in this paper is a minimisation problem subject to equality constraints, the derivation of the cautious Riccati solution is based on defining a set of Lagrange multipliers. The cautious Riccati solution is implemented to linear multi dimensional control problem and compared to the certainty equivalent Riccati solution.

On the lower semicontinuous quasiconvex envelope for unbounded integrands (I)

Motivated by the study of multidimensional control problems of Dieudonné-Rashevsky type, we raise the question how to understand to notion of quasiconvexity for a continuous function f with a convex body K instead of the whole space as the range of definition. In the present paper, we trace the consequences of an infinite extension of f outside K, and thus study quasiconvex functions which are allowed to take the value +∞. As an appropriate envelope, we introduce and investigate the lower semicontinuous quasiconvex envelope quasiconvex and lower semicontinuous, Our main result is a representation theorem for which generalizes Dacorogna's well-known theorem on the representation of the quasiconvex envelope of a finite function. The paper will be completed by the calculation of in two examples.

Pontryagin’s Maximum Principle for Multidimensional Control Problems in Image Processing

In the present paper, we prove a substantially improved version of the Pontryagin maximum principle for convex multidimensional control problems of Dieudonne-Rashevsky type. Although the range of the operator describing the first-order PDE system involved in this problem has infinite codimension, we obtain first-order necessary conditions in a completely analogous form as in the one-dimensional case. Furthermore, the adjoint variables are subjected to a Weyl decomposition. We reformulate two basic problems of mathematical image processing (determination of optical flow and shape from shading problem) within the framework of optimal control, which gives the possibility to incorporate hard constraints in the problems. In the convex case, we state the necessary optimality conditions for these problems.